django

Fix Django “Illegal mix of collations”

I have json file contains text non ASCII such kanji, tagalog etc.

{
    "Peru": ["S.A. - Sociedad Anónima", "S.A.A. - Sociedad Anónima Abierta"],

    "Philippines": ["Co. - Company", "Coop. - Cooperative", "Corp. - Corporation", "Ent. - Enterprise", "Inc. - Incorporated", "LLC - Limited Liability Company", "Ltd. - Limited", "Ltd. Co. - Limited Company", "Cía - Compañía", "SA - Sociedad Anónima"],

    "Poland": ["jednoosobowa działalność gospodarcza", "Przedsiębiorstwo Państwowe", "S.A. - spółka akcyjna", "s.c. - spółka cywilna", "S.K.A. - spółka komandytowo-akcyjna", "sp.j. - spółka jawna", "sp.k. - spółka komandytowa", "sp.p. - spółka partnerska", "Sp. z o.o. - spółka z ograniczoną odpowiedzialnością", "Spółdzielnia"],

    "Portugal": ["CRL - Cooperativa de Responsabilidade Limitada", "S.A. - Sociedade Anónima", "S.A. - Sociedade Aberta", "S.F. - Sociedade Fechada", "Lda. - Limitada", "Unipessoal Lda.", "SGPS - Sociedade Gestora de Participações Sociais"],

    "Romania": ["SNC - Societatea în nume colectiv", "SCS - Societatea în comandită simplă", "SCA - Societatea în comandită pe acțiuni", "SA - Societatea pe acțiuni", "SRL - Societatea cu răspundere limitată", "SRL cu proprietar unic - Societatea cu răspundere limitată cu proprietar unic", "S.A. - Societate pe Acţiuni", "S.C.A. - societate în comandită pe acţiuni", "S.C.S. - societate în comandită simplă", "S.N.C. - societate în nume colectiv", "S.R.L. - societate cu răspundere limitată", "PFA - persoana fizica autorizata", "O.N.G. - Organizație Non-Guvernamentală"],

    "Rusia": ["Nekommercheskaya organizatsiya/некоммерческая организация", "GP/ГП, GUP/ГУП - Gosudarstvennoye unitarnoye predpriyatie/Государственное унитарное предприятие", "IP/ИП - Individualny predprinimatel/Индивидуальный предприниматель", "OOO - Obshchestvo s ogranichennoy otvetstvennostyu/Общество с ограниченной ответственностью", "ПAO - Publichnoye aktsionernoye obshchestvo/Публичное акционерное общество", "kooperativ/кооператив", "AO - Aktsionernoye obshchestvo/Акционерное общество", "Prostoye Tovarishestvo - general partnership", "Kommanditnoe Tovarishestvo - limited partnership", "Hozyaystvennoe Partnerstvo - business partnership"],

    "Saudi Arabia": ["شركة ذات مسئولية محدودة - Private Limited Company", "شركة مساهمة - Joint-Stock company", "شركة تضامن - General Partnership Company", "شركة التوصية البسيطة - Limited Partnership", "شركة أجنبية - Foreign Company", "مؤسسة فردية - Individual Establishment"],
}

And I’m creating a script to read those data and insert into my database.
I’m using Django using MariaDB as my database and I found this error while inserting to the database.

django.db.utils.OperationalError: (1267, "Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='")

After several minutes find a solution, the issue occurs because python using unicode string but my database doesn’t.

To fix this issue, I just need to update my database table collation in shell following below.

ALTER TABLE my_table_name CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;

Reference: https://stackoverflow.com/a/7083511/1936697

Advertisements

Add HTTP_REFERER in Django Test Client

I’m testing my views in my django project and got “None” error because my view returning redirect to “HTTP_REFERER” that I get from HTTP Header.

In django test client apparently all request does not contain any HTTP header.
Here is my views:

class DeleteCategoryView(TemplateView):

    def post(self, request, *args, **kwargs):
        ...
        return redirect(request.META.get('HTTP_REFERER'))

And here is how I add “HTTP_REFERER” in my test client.

def test_delete_view(self):
    url = reverse('category:url-adm-delete')
    redirect_url = reverse('category:url-adm-category')
    response = self.client.post(path=url, data={'id': instance.pk}, HTTP_REFERER=redirect_url)
    self.assertRedirects(response, index_url)

Reference: http://stackoverflow.com/a/11819426/1936697

Django Bootstrap Pagination Helper

I Just writing simple script for creating simple pagination with django and bootstrap.
To use these, what you need to do just add the middleware class to your settings.py, within you views just pass your queryset to the paginate function, then just include the html to you template.

Here is the usage example:

# views.py
class MemberView(TemplateView):
    template_name = 'persons/list.html'
    
    def get(self, request, *args, **kwargs):
        person_list = Person.objects.all()
        members = paginate(request=request, queryset=person_list, limit=30)

        context = self.get_context()
        context.update({
            'members': members
        })
        return super(MemberView, self).render_to_response(context)

# list.html
{% include 'paging.html' with paginator=members %}

NB: I assumed you are using twitter bootstrap on your project.

Custom Validator in Django Rest Framework Serializer

Here is how to create your own validator for your serializer fields.
For example I have serializer that it will accept image not large than 20MB.
There are 2 approach for this, we can use function or class based.

class ProfileSerializer(serializers.Serializer):
    profile_pict = serializers.ImageField()

Function based:

def validate_20mb_size(value):
    max_size = 1024 * 1024 * 20  # 20MB
    if value.size > max_size:
        raise serializers.ValidationError('Profile Image too large.')

# Then we use like this.
profile_pict = serializers.ImageField(validators=[validate_20mb_size])

Class based:

class ImageSizeValidator(object):
     def __init__(self, max_size):
        self.max_size = 1024 * 1024 * max_size

     def __call__(self, value):
        if value.size > self.max_size:
            raise serializers.ValidationError('Profile Image too large.')

# Then we use like this.
profile_pict = serializers.ImageField(validators=[ImageSizeValidator(max_size=20)])

You can use whatever you want depend on your needs, but I prefer to use class based because we can use the constructor to make the validator more generic and also inheritance. 😀

Reference: http://www.django-rest-framework.org/api-guide/validators/#writing-custom-validators