Python

Python: Fix getting “about:blank” in Selenium with PhantomJS

Having an issue with selenium when loading some URL with selenium using PhantomJS.
When I get the current_url some URL returned “about:blank” instead of the valid URL.

After several workarounds, the issue because PhantomJS by default using “SSLv3” and many websites had a bug in SSL migrate to TLS and caused got a blank page.

Here is how I solve my issue.

from selenium import webdriver

driver = webdriver.PhantomJS(service_args=['--ignore-ssl-errors=true', '--ssl-protocol=any'])

Reference: http://stackoverflow.com/a/34257642/1936697

Advertisements

Python: Fix “OSError: [Errno 9] Bad file descriptor” in Selenium using PhantomJS

I’m having an issue when closing selenium driver and sometimes the selenium driver failed to quit. Here is how to I solve this issue.

import signal
from selenium import webdriver

driver = webdriver.PhantomJS()
# Some code goes here
driver.service.process.send_signal(signal.SIGTERM)
driver.quit()

Reference: http://stackoverflow.com/a/38493285/1936697

Create Python Class On The Fly

I just found a cool trick in python to create a class dynamically on the fly without creating the class definition.

The story behind this, I’m using Django in my project and I need to create serializer class dynamically based on user’s input. Here is how I solving my problem by creating serializer class on the fly.

def serializer_factory(list_context):
    fields = {}
    for i in list_context:
        fields[i] = serializers.CharField(required=True)
    GeneratedClass = type("SerializerFactory", (serializers.Serializer,), fields)
    return GeneratedClass

And here is how I used it.

context = ['name', 'address', 'current_location']
PeopleClass = serializer_factory(context)
serializer = PeopleClass(data=request.data)
serializer.is_valid()
"""
code above equivalent with.

class SerializerFactory(serializers.Serializer):
    name = serializers.CharField(required=True)
    address = serializers.CharField(required=True)
    current_location = serializers.CharField(required=True)
"""

“type” in python commonly uses in meta programming, and here is the detail for creating class using this keyword.

type("", (), {})

1. The first String argument is for defining class name.
2. Second argument is tuple to define your base class.
3. Third argument is dictionary for defining class property.

Create Empty Object in Python

I need to create dummy object in python for testing purpose. Here is how to create dummy object in python on the fly using “type”.

request = type('request', (), {})()  # Create request object on the fly
print request is None                # False
print request                        # <__main__.request at 0x1073f8590>

Reference: http://stackoverflow.com/a/19476841/1936697

Fix “command ‘gcc’ failed with exit status 4” Once Installing lxml

Today I’m installing sentry on my VPS, I’m using ubuntu 14.04 on smallest instance on DigitalOcean. But the installation failed because of sentry depend on lxml and the lxml failed to install through python pip. Here is the error stacktrace.

cc: internal compiler error: Killed (program cc1)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
    Compile failed: command 'gcc' failed with exit status 4
    cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /tmp/xmlXPathInitRJR8OE.c -o tmp/xmlXPathInitRJR8OE.o
    cc tmp/xmlXPathInitRJR8OE.o -lxml2 -o a.out
    error: command 'gcc' failed with exit status 4

I don’t know what exactly going on with this, but after looking up on stackoverflow. Some answer told that this is memory problem, to fix this we just need to increase our memory by adding swapfile. Here is how to fix this.

sudo dd if=/dev/zero of=/swapfile bs=1024 count=524288
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

Now I could install sentry without any trouble with this lxml issue.

Reference: http://stackoverflow.com/questions/24455238/lxml-installation-error-ubuntu-14-04-internal-compiler-error

Save Configuration in Python

Python has “ConfigParser” module that allow you read or write configuration to “.ini” file.
And here is how to read or write to config file. First of all you need to create “.ini” file, for instance “hello.ini”

[main]
username = myusername
password = mypassword

Read config file:

from ConfigParser import SafeConfigParser

config = SafeConfigParser()
config.read('hello.ini')

db_user = config.get('main', 'username')
db_pass = config.get('main', 'password')

Write config file:

from ConfigParser import SafeConfigParser

config = SafeConfigParser()
config.read('hello.ini')
config.add_section('main')
config.set('main', 'username', 'mynewusername')
config.set('main', 'password', 'mynewpassword')
config.set('main', 'new_key', 'my new value')

with open('hello.ini', 'w') as f:
    config.write(f)

Reference: http://stackoverflow.com/a/19078712/1936697

Atom for Python Development

Here is atom plugins that I used for my python development, because I’m no longer using PyCharm. 😀

hyperclick: https://atom.io/packages/hyperclick
autocomplete-python: https://atom.io/packages/autocomplete-python
linter: https://github.com/atom-community/linter
linter-flake8: https://github.com/AtomLinter/linter-flake8

For installation plugins above, you can refer to link that I have provided, because the docs pretty straightforward. 😀

Addition, maybe previously you are using sublime text and you are using gitgutter, you can add this snippet https://gist.github.com/Keda87/118f19c78286ca845d4d to your atom config to highlight all your changes on your project. Kinda look like gitgutter. 😀