Django

Handle Base64 Encoded Binary File in DRF

I’m writing a simple class for DRF field for handling base64 string for attachment both image and file. I’m using DRF 3 at this case.

And here is how to use it following below.

class MyModelSerializer(serializers.ModelSerializer):
    company_logo = Base64CharField(required=False)

    class Meta:
        model = MyModel
        fields = '__all__'

Feel free to use and improve my code. 😀

Add HTTP_REFERER in Django Test Client

I’m testing my views in my django project and got “None” error because my view returning redirect to “HTTP_REFERER” that I get from HTTP Header.

In django test client apparently all request does not contain any HTTP header.
Here is my views:

class DeleteCategoryView(TemplateView):

    def post(self, request, *args, **kwargs):
        ...
        return redirect(request.META.get('HTTP_REFERER'))

And here is how I add “HTTP_REFERER” in my test client.

def test_delete_view(self):
    url = reverse('category:url-adm-delete')
    redirect_url = reverse('category:url-adm-category')
    response = self.client.post(path=url, data={'id': instance.pk}, HTTP_REFERER=redirect_url)
    self.assertRedirects(response, index_url)

Reference: http://stackoverflow.com/a/11819426/1936697

Queryset Optimization Case Study 1

Today I’m revisit my legacy work that I have done long time a go. I’m developing admin panel that displaying list of product and including display total income that we have got in each items.

Below are my model schema:

class Item(models.Model):
    ...
    brand = models.ForeignKey(Brand)
    owner = models.ForeignKey(User)
    base_price = models.ForeignKey(BasePricing)
    categories = models.ManyToManyField(Category)

    @property
    def total_rental_income(self):
        """Getting total income."""
        return self.booking_set.aggregate(total=Sum('order__fee'))

class Booking(models.Model):
    ...
    item = models.ForeignKey(Item)

class Order(models.Model):
    ...
    booking = models.ForeignKey(Booking)
    fee = models.DecimalField(max_digits=12, decimal_places=2, default=0.00)

Currently I just select the data with simple queryset and display 50 items per page.
For your information, I’m profiling my queryset using django-querycount and django-debug-toolbar.

items = Item.objects.all()

Here is my query count:

|------|-----------|----------|----------|----------|------------|
| Type | Database  |   Reads  |  Writes  |  Totals  | Duplicates |
|------|-----------|----------|----------|----------|------------|
| RESP |  default  |   314    |    0     |   314    |     43     |
|------|-----------|----------|----------|----------|------------|
Total queries: 314 in 40.3557s

And here is the detail duplicate query that captured by django debug toolbar.
screen-shot-2016-12-30-at-5-01-10-pm

Now I realized I have to optimize this queryset. My first attempt to fix this issue are using “select_related” all foreign keys and “prefetch_related” my many to many field to select the data eagerly. So my queryset will load all at once instead of the lazy load (default django orm behavior).

items = Item.objects.all().select_related(
    'brand', 'owner', 'base_price'
).prefetch_related(
    'categories'
)

Here is the query count result after my first optimization.

|------|-----------|----------|----------|----------|------------|
| Type | Database  |   Reads  |  Writes  |  Totals  | Duplicates |
|------|-----------|----------|----------|----------|------------|
| RESP |  default  |    74    |    0     |    74    |     9      |
|------|-----------|----------|----------|----------|------------|
Total queries: 74 in 18.0481s

According the query count result, the query much better than before. 314 reads reduced into 74 reads and was 43 duplicates reduced into 9 duplicates.
I did not attach my django debug toolbar because previous captured queries already disappeared. But there’s still one thing that still duplicate from my model schema above. 50 duplicates in my template.
screen-shot-2016-12-30-at-5-22-10-pm

I just know this, that I called my custom property from “Item” model in my template and this custom property is always evaluated and not affected with our optimization above.

Now, here is my second optimization to fix my aggregation issue. I remove my property in my “Item” model and uses “annotate” in my queryset.

items = Item.objects.all().select_related(
    'brand', 'owner', 'base_price'
).prefetch_related(
    'categories'
).annotate(
    rental_income=Sum('booking__order__fee')
)

# And I call my income calculation within my template loop.
{{ item.rental_income|default:0 }}

And here is my query count result.

|------|-----------|----------|----------|----------|------------|
| Type | Database  |   Reads  |  Writes  |  Totals  | Duplicates |
|------|-----------|----------|----------|----------|------------|
| RESP |  default  |    18    |    0     |    18    |     9      |
|------|-----------|----------|----------|----------|------------|
Total queries: 18 in 11.4635s

Now my queryset way better than before, 74 queries reduced into 18 queries.
I still had 9 duplicate query and I’m still working on it. Maybe i’ll update this post if I can optimize the whole queryset. 😀

References:
https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-related
https://docs.djangoproject.com/en/dev/ref/models/querysets/#prefetch-related
https://docs.djangoproject.com/en/dev/topics/db/aggregation/#aggregation

Django: Invalidate Cached Property

Sometimes we create property within our model for utility helper. And django provide decorator “@cached_property” for caching heavy computation within our property.

class Member(models.Model):
    # field definitions.

    @cached_property
    def score(self):
        return # heavy computation/query

I got a problem once testing my model and the property is still load the cached version of data instead of the latest. Here is how to invalidate cached property.

del member.score          # Invalidate cache.
delattr(member, 'score')  # Alternative to invalidate cache.

Reference: https://docs.djangoproject.com/en/dev/ref/utils/#django.utils.functional.cached_property

Remove “required” Attribute on Django 1.10 Forms

I just starting new project with latest version of Django 1.10, the most basic feature and incredibly amazing is django form, that could simplify form rendering and tied up with python object including the validation.

If you realized, django 1.10 has new mechanism for doing input validation. If you are get used using django before version 1.10, all input will validated on server side and throw error message to template, if the input invalid. But since django 1.10, all validation will validated on template first with html5’s “required” attribute by default then it will validated on backend views.

Here is the changelog documentation: https://docs.djangoproject.com/en/1.10/ref/forms/api/#django.forms.Form.use_required_attribute

Here is how to disable this behavior, by removing “required” attribute on rendered form fields.

form = MyForm(use_required_attribute=False)

It’s a bit tricky if you are using class based views with FormView.

class MyFormView(FormView):
    form_class = MyForm
    template_name = 'create_form.html'

    def get_form_kwargs(self):
        kwargs = super(MyFormView, self).get_form_kwargs()
        kwargs['use_required_attribute'] = False
        return kwargs