Chozabu Chozabu - 2 months ago 17x
Python Question

Should I take steps to ensure a Django app can scale before writing it?

So, I'm looking at writing an app with python2 django(-rest-framework), postgres and angular.

I'm aware there are lots of things that can be done

  • multi-server setup behind load balancer

  • DB replication/sharding?

  • caching (in various ways)

  • swapping DRF serialiser for serpy

  • running on python3

  • running on pypy

my question is - Which of these (or other things) should really be done right at the start of the project?


Write with scalability in mind.

scaliability is not limited just to production servers/environments but also to development environments.

Always write with scalability in mind.

At development

Scalability at development let you develop the product seemlessly.

  1. Structure your repository
    Use git branching models like GitFlow so that developers can work on parallel, or a single developer can switch working on different features. Use a bug tracker.

  2. Design your apps.
    Before actually writing a single line of code, write down what apps you are going to write. Design apps as to minimize Relations (ManyToMany, ForeignKey, etc..), imports. Django offers pulggable app architecture feel free to use it wisely.

  3. Write your tests first.
    This ensures that you can migrate(production environments), upgrade and downgrade with less pain and hairloss. Trust me writing tests feels so boring, but it is worth to.

  4. Abstract models, managers
    Use Abstract Models and Managers, it can eliminate bolier plate model code and help you maintain the code.

  5. Name variables, classes and methods descriptive.
    Name the variables, classes and methods descriptive, as you would be able to know what it represents without looking documentation.

  6. Document code.
    Feel free to document classes and methods, so you or other peers who look into the code get an idea of what it is indented for than stacktracing to see what a method is doing.

  7. Use debug toolbar
    Use django debug toolbar on developement, as you test your API, use prefectch_related() and select_related() to minimize/eliminate duplicated queries.

  8. Modularize code.
    Modularize the code. Python and django in general encourage use of modules. Modules are easy to manage. Use classes, more inheritance and abstract base classes to reuse code.

  9. Use Continuous Integration
    use continuous integration test your repo and make sure new pushes don't break the system.

At production.

scalability at production let you serve the product to infinite users seemlessly.

  1. For multi server setup

    • Stick with rest design principles.
    • Eliminate sessions.
    • Use distributed cache like Redis.
  2. Swapping DRF serializer for serpy
    Start with serpy if you need more speed and if you are comfortable with. It is better to stick with Serpy than rewriting DRF serializer as writing both looks smiliar, but make sure you are not wasting time by optimizing for the lost 1 or 2ms.

  3. Running on python3
    Depends on the libraries that you plan to use.

  4. Running on pypy
    pypy is faster than the standard implementations. It depends on the library compatibility to use pypy. A list of compatibile package and status of compatibility.

now the question,

Which of these (or other things) should really be done right at the start of the project?

Ans: Developemet (1,2,3,4,5,6,7,8) Production(1,2)