Why i chose Django for my next project – Introducing my largest failed side project

So today I was scratching my head for good three hours over Django REST Framework.

While I furiously googled possible solutions, I kept asking why do I torture myself with this complicated framework?

Don’t get me wrong.

Django is a great web development framework.

But is very opinionated and has a very steep learning curve.

I started learning Django three years ago in 2016 after I finished my now failed large side project.

Website, which provides risk analytic coverage of European and US stocks.

The project was hacked together with WordPress (! 😱) , PHP, Python and cron jobs.

The core analytics were done in Python, which fed an FTP server with daily stock data, scheduled with windows task scheduler.

On the server side, I had a cron, that executed a PHP script, that loaded batch files into WordPress database.

Sounds hacky right?

Yeah. I didn’t know how to code back then.

Now, that I know a little better, I admire myself how I came up with such overcomplicated architecture.

After my project went live, I realized, that there is no way I can scale and improve that architecture.

So, I looked over the internet for web backend frameworks, that allows me to execute python scripts natively.

I found out about Flask and Django.

My first impression on Flask was positive.

Simple framework for quick prototypes, but difficult for going full stack, as it didn’t provide any support for functionalities, that WordPress provided out of the box, such as user auth or basic templating system.

Django offered much more.

Integrated user management, authentication, REST framework, MVC approach, basic templating and routing system.

I was excited.

I can finally develop apps, that can incorporate machine learning and super complex data analytic tasks natively.

But then I read Django official documentation.

I read it again, and again…and many time over again.

My mind was overwhelmed.

The difficulty was definitely 1000x higher than learning how to deploy WordPress blog and hack WordPress template with custom PHP snippets.

The best way to learn is to try it myself and fail.

So, I tried to follow official Django documentation and create basic todo app according to their step by step guide.

I immediately identified many new concepts, that require significant change of the mindset.

Looking from the perspective of a non-programmer person, who grew up on WAMP stack (Apache, MySQL, PHP).

Deployment

There is no one-click installation program to setup whole stack on your hosting.

You have to figure out by yourself how to set up and configure Nginx and Gunicorn.

Database management (Model)

Because Django uses ORM, whole database model is defined in models file.

You just define table structure, columns, their data types and relationships.

All changes are deployed by calling two commands – make migration and migrate.

No more fiddling with MyPHPAdmin. Heck yeah!

View

A new concept for me.

Views are files where you define your app logic.

That includes data extraction from database with ORM or raw SQL queries.

Data manipulation like conditional filtering, sorting, parsing.

Finally you return processed data to Django template for rendering.

Templates

A refreshing way of creating HTML output.

I don’t have to write tons of code to do simple loops and variable transformations like in PHP.

I still don’t know how to efficiently develop dynamic pages using the Django templating system, but I believe it will be less painful than with PHP.

Url Routing

Also a new concept.

You tie predefined URL paths to views and templates.

Each URL redirects you to Django template and each template is filled with data from the corresponding view.

Everything is tightly coupled together.

I guess it has to be. Otherwise, it wouldn’t be server-side rendering.

REST Framework

The only thing, that works as expected.

There is no need to learn additional concepts.

Pull data from the model to view.

Tie view to URL and define JSON object structure in Django serializer file.

Conclusion

The architecture works out of the box if you use it as Django documentation tells you to use it – as simple CRUD application.

But simplicity ends when you start to have more complex models, that require heavy data transformation.

Then you have to figure out how to do complex query sets with multiple conditional joins, subqueries, with constructions and aggregation.

Despite all the quirks and hurdles.

There is no other option if you want to use analytical python libraries in your app.

That’s why I didn’t give up on Django and started to learn it again 3 months ago.

It’s painful experience, but I have to sacrifice prototyping speed for long term scalability of the project.

Furthermore, once I finish my first Django web application, then I will be able to create anything I want in the future.

That’s basic value proposition of Django framework – promise and hope, that once I overcome the steep barrier of entry, it will give me new powers and open new possibilities.

To end this article, I should say something more about my failed side project.

It wasn’t technological debt, that killed it.

It supposed to solve my problem with tracking hundreds of Stocks daily, calculate their risk measures, compare them side by side and watch for any news articles, that appears.

I thought other people have the same problem, but I was wrong.

My risk measures were too advanced to comprehend.

Ordinary inexperienced stock traders just need actual price, moving average and nothing else.

Simply if you don’t understand something, you won’t believe in it.

So, I failed to get initial traction from early adopters.

Eventually, faith sealed my project forever as my only source of Stock data – Yahoo finance API – closed their shop so my website stopped updating itself with new stock data.

But I learned a lot in the process – hacking WordPress templates, working with HighCharts JS, scripting in python, scripting in PHP.

Mainly I didn’t give up and finished my first large side project and its pain points led me to Django 😎

Share it!


Want to support me in my journey πŸš€? You can send me a donation below πŸ‘‡.

Donate with Paypal

Leave a Reply

avatar

Want to support me in my journey πŸš€? You can send me a donation below πŸ‘‡.

Donate with Paypal

Made with πŸ”₯ by Viet Phan 2018