Django Server Sent Event Tutorial

Server Sent Event atau biasa disebut SSE merupakan teknologi dalam pengembangan web dimana client kita dapat menerima push message dari server melalui protokol HTTP tanpa melakukan request. Sedikit berbeda dengan konsep web pada umumnya, dimana server akan mengirimkan response apabila menerima request dari client.

Studi Kasus

Sebuah website youtube downloader, dimana user akan memasukkan youtube URL, kemudian akan di proses secara background dikarenakan proses download file yang lama dan Django akan mengirimkan notifikasi/event kepada client saat download file telah selesai.

Continue reading “Django Server Sent Event Tutorial”

Konfigurasi Gunicorn Pada Django

Perlu dipahami apabila kita membuat sebuah aplikasi berbasis web, kita harus tau karakteristik dari aplikasi yang kita buat. Apakah aplikasi kita banyak menangani CPU bound atau I/O bound?

Berikut definisi singkat dan contoh dari masing-masing karakteristik tersebut.

  • I/O Bound.
    Aplikasi terfokus untuk melakukan proses input dan output.
    Contohnya seperti aplikasi website pendaftaran, web scrapper dan html to pdf converter.
  • CPU Bound.
    Aplikasi terfokus untuk melakukan proses komputasi.
    Contohnya seperti aplikasi yang melakukan proses machine learning, encoding video dan dashboard perhitungan statistik.

2 karakteristik ini juga memiliki konfigurasi yang berbeda agar dapat berjalan secara optimal. Kali ini saya ingin menuliskan catatan beberapa konfigurasi Gunicorn dan semoga mudah dipahami.

Dalam pembuatan aplikasi web dengan performa yang baik, sangat berkaitan dengan yang namanya concurrency. Ada 3 jenis concurrency yang harus kita pahami dalam konfigurasi Gunicorn.

Continue reading “Konfigurasi Gunicorn Pada Django”

Asynchronous Pada Django Views

Langkah besar baru saja terjadi pada Framework Django, dimana fitur async views, async middleware dan async test client baru saja ditambahkan kedalam codebase Django pada pull request ini.

Mungkin bagi pengguna Django sebelumnya  sudah terbiasa dengan kode yang berjalan secara synchronous karena framework ini dibuat dimasa python belum support kode asyncronous.

Dengan adanya modul asyncio pada python 3, Django Maintainer  sepakat untuk menambahkan fitur async pada Django agar kita semua bisa memanfaatkan asyncio secara maksimal pada saat melakukan pembuatan aplikasi menggunakan Django.

Continue reading “Asynchronous Pada Django Views”

Fix Django-Jet Select Dropdown not Working on Custom Page

I’m having an issue when creating a custom page in django-jet. I’m using v1.0.6 and always got jQuery error in dropdown select in my custom page.

Uncaught TypeError: django.jQuery is not a function
at HTMLSelectElement.<anonymous> (bundle.min.js?v=1.0.8:1)
at HTMLSelectElement.dispatch (bundle.min.js?v=1.0.8:9)
at HTMLSelectElement.m.handle (bundle.min.js?v=1.0.8:9)
at Object.trigger (bundle.min.js?v=1.0.8:9)
at HTMLSelectElement.<anonymous> (bundle.min.js?v=1.0.8:9)
at Function.each (bundle.min.js?v=1.0.8:8)
at ot.fn.init.each (bundle.min.js?v=1.0.8:7)
at ot.fn.init.trigger (bundle.min.js?v=1.0.8:9)
at n.select (bundle.min.js?v=1.0.8:12)
at o.<anonymous> (bundle.min.js?v=1.0.8:12)

I solved my issue by adding jquery manually following below in my custom page.

{% block extrahead %}
{{ block.super }}
<script type="text/javascript" src="{% static 'admin/js/vendor/jquery/jquery.js' %}"></script>
<script type="text/javascript" src="{% static 'admin/js/jquery.init.js' %}"></script>
{% endblock %}

Hope this issue fixed in next release.

Reference: https://github.com/geex-arts/django-jet/issues/161#issuecomment-412071688

Running Specific Test in django-nose

Since I used django-nose for my project. Django-nose runner cannot behave like default django test runner.

Following command will not work in django-nose test runner.

# Run all the tests found within the 'animals' package
$ ./manage.py test appname

# Run just one test case
$ ./manage.py test appname.tests.TestCaseName

# Run just one test method
$ ./manage.py test appname.tests.TestCaseName.test_function_name

Because django-nose structured all the test in one package called tests following below.

tests:
   |- __init__.py
   |- test_views.py
   |- test_forms.py
   |- test_endpoints.py
   |- test_utils.py

Here is the alternative to run specific task in django-nose.

# Run all the tests found within the 'animals' package
$ ./manage.py test appname

# Run just one test case
$ ./manage.py test appname.tests.test_script:TestCaseName

# Run just one test method
$ ./manage.py test appname.tests.test_script:TestCaseName.test_function_name

Reference: https://stackoverflow.com/a/18834222/1936697

Exclude folders and line of codes from Test Coverage

Here is my coverage setup in my project. I excluded some folders and line of codes that not important in my testing.

I put these config in .coveragerc

[run]
omit =
    */urls/*
    */migrations/*
    */tests/*
    */__init__.py
    */tests.py
    */apps.py
    */rss.py
    */urls.py

[report]
exclude_lines =
    def __unicode__
    if setttings.TESTING
    def __repr__
    def __str__

Disable django-haystack During Testing

I use django-haystack and elasticsearch for the search engine in our apps. But I don’t want my model that registered to haystack do indexing during the testing. I avoid haystack indexing by adding these following lines of code below in my settings.py.

import sys

TESTING = 'test' in sys.argv
if TESTING:
    HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.BaseSignalProcessor'
    HAYSTACK_CONNECTIONS['default'] = {
        'ENGINE': 'haystack.backends.simple_backend.SimpleEngine',
    }

Reference: https://stackoverflow.com/q/33108569/1936697

Fix Django “Illegal mix of collations”

I have json file contains text non ASCII such kanji, tagalog etc.

{
    "Peru": ["S.A. - Sociedad Anónima", "S.A.A. - Sociedad Anónima Abierta"],

    "Philippines": ["Co. - Company", "Coop. - Cooperative", "Corp. - Corporation", "Ent. - Enterprise", "Inc. - Incorporated", "LLC - Limited Liability Company", "Ltd. - Limited", "Ltd. Co. - Limited Company", "Cía - Compañía", "SA - Sociedad Anónima"],

    "Poland": ["jednoosobowa działalność gospodarcza", "Przedsiębiorstwo Państwowe", "S.A. - spółka akcyjna", "s.c. - spółka cywilna", "S.K.A. - spółka komandytowo-akcyjna", "sp.j. - spółka jawna", "sp.k. - spółka komandytowa", "sp.p. - spółka partnerska", "Sp. z o.o. - spółka z ograniczoną odpowiedzialnością", "Spółdzielnia"],

    "Portugal": ["CRL - Cooperativa de Responsabilidade Limitada", "S.A. - Sociedade Anónima", "S.A. - Sociedade Aberta", "S.F. - Sociedade Fechada", "Lda. - Limitada", "Unipessoal Lda.", "SGPS - Sociedade Gestora de Participações Sociais"],

    "Romania": ["SNC - Societatea în nume colectiv", "SCS - Societatea în comandită simplă", "SCA - Societatea în comandită pe acțiuni", "SA - Societatea pe acțiuni", "SRL - Societatea cu răspundere limitată", "SRL cu proprietar unic - Societatea cu răspundere limitată cu proprietar unic", "S.A. - Societate pe Acţiuni", "S.C.A. - societate în comandită pe acţiuni", "S.C.S. - societate în comandită simplă", "S.N.C. - societate în nume colectiv", "S.R.L. - societate cu răspundere limitată", "PFA - persoana fizica autorizata", "O.N.G. - Organizație Non-Guvernamentală"],

    "Rusia": ["Nekommercheskaya organizatsiya/некоммерческая организация", "GP/ГП, GUP/ГУП - Gosudarstvennoye unitarnoye predpriyatie/Государственное унитарное предприятие", "IP/ИП - Individualny predprinimatel/Индивидуальный предприниматель", "OOO - Obshchestvo s ogranichennoy otvetstvennostyu/Общество с ограниченной ответственностью", "ПAO - Publichnoye aktsionernoye obshchestvo/Публичное акционерное общество", "kooperativ/кооператив", "AO - Aktsionernoye obshchestvo/Акционерное общество", "Prostoye Tovarishestvo - general partnership", "Kommanditnoe Tovarishestvo - limited partnership", "Hozyaystvennoe Partnerstvo - business partnership"],

    "Saudi Arabia": ["شركة ذات مسئولية محدودة - Private Limited Company", "شركة مساهمة - Joint-Stock company", "شركة تضامن - General Partnership Company", "شركة التوصية البسيطة - Limited Partnership", "شركة أجنبية - Foreign Company", "مؤسسة فردية - Individual Establishment"],
}

And I’m creating a script to read those data and insert into my database.
I’m using Django using MariaDB as my database and I found this error while inserting to the database.

django.db.utils.OperationalError: (1267, "Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='")

After several minutes find a solution, the issue occurs because python using unicode string but my database doesn’t.

To fix this issue, I just need to update my database table collation in shell following below.

ALTER TABLE my_table_name CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;

Reference: https://stackoverflow.com/a/7083511/1936697

Handle Base64 Encoded Binary File in DRF

I’m writing a simple class for DRF field for handling base64 string for attachment both image and file. I’m using DRF 3 at this case.

And here is how to use it following below.

class MyModelSerializer(serializers.ModelSerializer):
    company_logo = Base64CharField(required=False)

    class Meta:
        model = MyModel
        fields = '__all__'

Feel free to use and improve my code. 😀