Django Server Sent Event Tutorial

Server Sent Event atau biasa disebut SSE merupakan teknologi dalam pengembangan web dimana client kita dapat menerima push message dari server melalui protokol HTTP tanpa melakukan request. Sedikit berbeda dengan konsep web pada umumnya, dimana server akan mengirimkan response apabila menerima request dari client.

Studi Kasus

Sebuah website youtube downloader, dimana user akan memasukkan youtube URL, kemudian akan di proses secara background dikarenakan proses download file yang lama dan Django akan mengirimkan notifikasi/event kepada client saat download file telah selesai.

Continue reading “Django Server Sent Event Tutorial”

Konfigurasi Gunicorn Pada Django

Perlu dipahami apabila kita membuat sebuah aplikasi berbasis web, kita harus tau karakteristik dari aplikasi yang kita buat. Apakah aplikasi kita banyak menangani CPU bound atau I/O bound?

Berikut definisi singkat dan contoh dari masing-masing karakteristik tersebut.

  • I/O Bound.
    Aplikasi terfokus untuk melakukan proses input dan output.
    Contohnya seperti aplikasi website pendaftaran, web scrapper dan html to pdf converter.
  • CPU Bound.
    Aplikasi terfokus untuk melakukan proses komputasi.
    Contohnya seperti aplikasi yang melakukan proses machine learning, encoding video dan dashboard perhitungan statistik.

2 karakteristik ini juga memiliki konfigurasi yang berbeda agar dapat berjalan secara optimal. Kali ini saya ingin menuliskan catatan beberapa konfigurasi Gunicorn dan semoga mudah dipahami.

Dalam pembuatan aplikasi web dengan performa yang baik, sangat berkaitan dengan yang namanya concurrency. Ada 3 jenis concurrency yang harus kita pahami dalam konfigurasi Gunicorn.

Continue reading “Konfigurasi Gunicorn Pada Django”

Asynchronous Pada Django Views

Langkah besar baru saja terjadi pada Framework Django, dimana fitur async views, async middleware dan async test client baru saja ditambahkan kedalam codebase Django pada pull request ini.

Mungkin bagi pengguna Django sebelumnya  sudah terbiasa dengan kode yang berjalan secara synchronous karena framework ini dibuat dimasa python belum support kode asyncronous.

Dengan adanya modul asyncio pada python 3, Django Maintainer  sepakat untuk menambahkan fitur async pada Django agar kita semua bisa memanfaatkan asyncio secara maksimal pada saat melakukan pembuatan aplikasi menggunakan Django.

Continue reading “Asynchronous Pada Django Views”

Fix Django “Illegal mix of collations”

I have json file contains text non ASCII such kanji, tagalog etc.

{
    "Peru": ["S.A. - Sociedad Anónima", "S.A.A. - Sociedad Anónima Abierta"],

    "Philippines": ["Co. - Company", "Coop. - Cooperative", "Corp. - Corporation", "Ent. - Enterprise", "Inc. - Incorporated", "LLC - Limited Liability Company", "Ltd. - Limited", "Ltd. Co. - Limited Company", "Cía - Compañía", "SA - Sociedad Anónima"],

    "Poland": ["jednoosobowa działalność gospodarcza", "Przedsiębiorstwo Państwowe", "S.A. - spółka akcyjna", "s.c. - spółka cywilna", "S.K.A. - spółka komandytowo-akcyjna", "sp.j. - spółka jawna", "sp.k. - spółka komandytowa", "sp.p. - spółka partnerska", "Sp. z o.o. - spółka z ograniczoną odpowiedzialnością", "Spółdzielnia"],

    "Portugal": ["CRL - Cooperativa de Responsabilidade Limitada", "S.A. - Sociedade Anónima", "S.A. - Sociedade Aberta", "S.F. - Sociedade Fechada", "Lda. - Limitada", "Unipessoal Lda.", "SGPS - Sociedade Gestora de Participações Sociais"],

    "Romania": ["SNC - Societatea în nume colectiv", "SCS - Societatea în comandită simplă", "SCA - Societatea în comandită pe acțiuni", "SA - Societatea pe acțiuni", "SRL - Societatea cu răspundere limitată", "SRL cu proprietar unic - Societatea cu răspundere limitată cu proprietar unic", "S.A. - Societate pe Acţiuni", "S.C.A. - societate în comandită pe acţiuni", "S.C.S. - societate în comandită simplă", "S.N.C. - societate în nume colectiv", "S.R.L. - societate cu răspundere limitată", "PFA - persoana fizica autorizata", "O.N.G. - Organizație Non-Guvernamentală"],

    "Rusia": ["Nekommercheskaya organizatsiya/некоммерческая организация", "GP/ГП, GUP/ГУП - Gosudarstvennoye unitarnoye predpriyatie/Государственное унитарное предприятие", "IP/ИП - Individualny predprinimatel/Индивидуальный предприниматель", "OOO - Obshchestvo s ogranichennoy otvetstvennostyu/Общество с ограниченной ответственностью", "ПAO - Publichnoye aktsionernoye obshchestvo/Публичное акционерное общество", "kooperativ/кооператив", "AO - Aktsionernoye obshchestvo/Акционерное общество", "Prostoye Tovarishestvo - general partnership", "Kommanditnoe Tovarishestvo - limited partnership", "Hozyaystvennoe Partnerstvo - business partnership"],

    "Saudi Arabia": ["شركة ذات مسئولية محدودة - Private Limited Company", "شركة مساهمة - Joint-Stock company", "شركة تضامن - General Partnership Company", "شركة التوصية البسيطة - Limited Partnership", "شركة أجنبية - Foreign Company", "مؤسسة فردية - Individual Establishment"],
}

And I’m creating a script to read those data and insert into my database.
I’m using Django using MariaDB as my database and I found this error while inserting to the database.

django.db.utils.OperationalError: (1267, "Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='")

After several minutes find a solution, the issue occurs because python using unicode string but my database doesn’t.

To fix this issue, I just need to update my database table collation in shell following below.

ALTER TABLE my_table_name CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;

Reference: https://stackoverflow.com/a/7083511/1936697

Add HTTP_REFERER in Django Test Client

I’m testing my views in my django project and got “None” error because my view returning redirect to “HTTP_REFERER” that I get from HTTP Header.

In django test client apparently all request does not contain any HTTP header.
Here is my views:

class DeleteCategoryView(TemplateView):

    def post(self, request, *args, **kwargs):
        ...
        return redirect(request.META.get('HTTP_REFERER'))

And here is how I add “HTTP_REFERER” in my test client.

def test_delete_view(self):
    url = reverse('category:url-adm-delete')
    redirect_url = reverse('category:url-adm-category')
    response = self.client.post(path=url, data={'id': instance.pk}, HTTP_REFERER=redirect_url)
    self.assertRedirects(response, index_url)

Reference: http://stackoverflow.com/a/11819426/1936697

Django Bootstrap Pagination Helper

I Just writing simple script for creating simple pagination with django and bootstrap.
To use these, what you need to do just add the middleware class to your settings.py, within you views just pass your queryset to the paginate function, then just include the html to you template.

Here is the usage example:

# views.py
class MemberView(TemplateView):
    template_name = 'persons/list.html'
    
    def get(self, request, *args, **kwargs):
        person_list = Person.objects.all()
        members = paginate(request=request, queryset=person_list, limit=30)

        context = self.get_context()
        context.update({
            'members': members
        })
        return super(MemberView, self).render_to_response(context)

# list.html
{% include 'paging.html' with paginator=members %}

NB: I assumed you are using twitter bootstrap on your project.

class PagingMiddleware(object):
"""Middleware to detect current page within page pagination."""
def process_request(self, request):
if 'page' in request.GET:
try:
request.active_page = int(request.GET.get('page'))
except ValueError:
request.active_page = 1
else:
request.active_page = 1
view raw middlewares.py hosted with ❤ by GitHub
<div class="row">
<div class="col-md-5 col-sm-5 lh-xl">
Showing {{ paginator.start_index }} to {{ paginator.end_index }} of {{ paginator.paginator.count }} records.
</div>
<div class="col-md-7 col-sm-7 pull-right">
<ul class="pagination">
<li class="{{ paginator.has_previous|yesno:',disabled' }}"><a href="?page=1"><i class="fa fa-angle-double-left"></i></a></li>
<li class="{{ paginator.has_previous|yesno:',disabled' }}"><a href="{% if paginator.has_previous %}?page={{ paginator.previous_page_number }}{% else %}#{% endif %}"><i class="fa fa-angle-left"></i></a></li>
{% for page in paginator.paginator.page_range %}
<li class="{% if request.active_page == page %}active{% endif %}"><a href="?page={{ page }}">{{ page }}</a></li>
{% endfor %}
<li class="{{ paginator.has_next|yesno:',disabled' }}"><a href="{% if paginator.has_next %}?page={{ paginator.next_page_number }}{% else %}#{% endif %}"><i class="fa fa-angle-right"></i></a></li>
<li class="{{ paginator.has_next|yesno:',disabled' }}"><a href="?page={{ paginator.paginator.num_pages }}"><i class="fa fa-angle-double-right"></i></a></li>
</ul>
</div>
</div>
view raw paging.html hosted with ❤ by GitHub
from django.core.paginator import Paginator, PageNotAnInteger, EmptyPage
def paginate(request, queryset=None, limit=10):
"""
Helper to transform given queryset to django paginator.
Params:
:request -- Django request object.
:queryset -- Model queryset.
:limit -- Limit record to be display.
"""
paginator = Paginator(queryset, limit)
page = request.GET.get('page')
try:
paginated = paginator.page(page)
except PageNotAnInteger:
paginated = paginator.page(1)
except EmptyPage:
paginated = paginator.page(paginator.num_pages)
return paginated
view raw utils.py hosted with ❤ by GitHub