My talk got selected for #Pycon India 2012

My proposed talk titled Develop for an international audience got selected for Pycon India, 2012. It’s time to start working on the slides. I am thinking to use rst to write my slides. Also, I have booked by flight tickets for Pycon 🙂

Thanks everyone who voted for my talk.

Advertisements

Validate Python string translation in Transifex

Transifex already supported validating translations of old styled Python strings, e.g.,

"A sample string with a %(keyword)s argument." % {'keyword': 'key word'}

The validation is done by checking if all the positional and keyword arguments are present in the translation string and the translation string does not contain any extra argument which is not in the source string. You can have a look at the validator code here.

However, the existing validator is not able to check for replacement fields in new style Python format strings, e.g.

"This is a sample string with different replacement fields: {} {1} {foo["bar"]:^30}".format(
"arg0", "arg1", foo={"bar":"a kwarg"})

I tried to devise a regex to extract the replacement fields in the Python format string based on the grammar defined here.

# Regex to find format specifiers in a Python string

import re

field_name = '(?P<field_name>(?P<arg_name>\w+|\d+){0,1}'\
                '(?:(?P<attribute_name>\.\w+)|'\
                '(?P<element_index>\[(?:\d+|(?:[^\]]+))\]))*)'
conversion = '(?P<conversion>r|s)'
align = '(?:(?P<fill>[^}{]?)(?P<align>[<>^=]))'
sign = '(?P<sign>[\+\- ])'
width = '(?P<width>\d+)'
precision = '(?P<precision>\d+)'
type_ = '(?P<type_>[bcdeEfFgGnosxX%])'
format_spec = ''\
    '(?P<format_spec>'\
        '%(align)s{0,1}'\
        '%(sign)s{0,1}#?0?'\
        '%(width)s{0,1},?'\
        '(?:\.%(precision)s){0,1}'\
        '%(type)s{0,1}'\
    ')' % {
        'align': align,
        'sign': sign,
        'width': width,
        'precision': precision,
        'type': type_
}
replacement_field = ''\
    '\{'\
    '(?:'\
        '%(field_name)s{0,1}'\
        '(?:!%(conversion)s){0,1}'\
        '(?:\:%(format_spec)s){0,1}'\
    ')'\
    '\}' % {
        'field_name': field_name,
        'conversion': conversion,
        'format_spec': format_spec
}

printf_re = re.compile(
    '(?:' + replacement_field + '|'
        '%((?:(?P<ord>\d+)\$|\((?P<key>\w+)\))?(?P<fullvar>[+#-]*(?:\d+)?'
            '(?:\.\d+)?(hh\|h\|l\|ll)?(?P<type>[\w%])))'
    ')'
)

Well, with the above, I was able to parse almost all the cases discussed here except for this one:

import datetime
d = datetime.datetime(2010, 7, 4, 12, 15, 58)
s = '{:%Y-%m-%d %H:%M:%S}'.format(d)

I was not sure how I could fit the above case to my regex. After some discussions in #python on IRC, I found some limitations of regular expressions and that it is not Turing complete. People suggested me to use some parser tools.

I, being a strong supporter of “Never re invent the wheel”, gave another shot to find some existing solution and lucky I was to come across _formatter_parser() of a Python string object.  It correctly found all replacement fields in python format strings properly and returned  an iterable of tuples (literal_textfield_nameformat_specconversion). All I needed then was to convert this info to a list of replacement fields in a format string. A simple script below would is all that I needed to extract replacement fields in a format string in Python:

replacement_fields = []
s = "{foo:^+30f} bar {0} foo {} {time:%Y-%m-%d %H:%M:%S}"

for literal_text, field_name, format_spec, conversion in \
        s._formatter_parser():
    if field_name is not None:
        replacement_field = field_name
        if conversion is not None:
            replacement_field += '!' + conversion
        if format_spec:
            replacement_field += ':' + format_spec
        replacement_field = '{' + replacement_field + '}'
        replacement_fields.append(replacement_field)
print replacement_fields
["{foo:^+30f}", "{0}", "{}", "{time:%Y-%m-%d %H:%M:%S}"]

That’s all. Simple and easy, isn’t it?

A year at Transifex

It’s more than a year now that I have been working at Transifex. It’s a great experience to be a part of the Transifex team. Well, it’s been a roller coaster ride for me at Transifex. I had to go through steep learning curves, work with new stuffs, deliver great features, meet strict deadlines. It was fun, because of being part of an awesome team. I am very much thankful to Apostolis, Konstantinos, John and Diego for guiding me and helping me.

During the course of this journey, I have made many friends. It’s always great to speak with the OpenTranslators folk. They all rock. I am also very much thankful to Youversion, Eventbrite, Pinterest, Dropbox folks for their queries and feedback. It makes me happy to help resolve issues with Transifex and add new features for our users. It gives me a sense of satisfaction that I am able to play my part for the greater good.

Mozilla has always fascinated me since a long time. I love Mozilla, it’s logos, it’s products and it’s goals. At Transifex, I worked on developing new API features to support Pontoon use Transifex as a backend. Following this, I also got involved with Pontoon’s development and have made a few commits. Finally, I contributed in some way to a Mozilla project.

A long road lies ahead. I hope it to be full of new challenges and excitement.

Updating all your fedora package git directories

Ankur's Tech blog

Most package maintainers already know this (or use something way easier), but this is how I go about updating all my fedora package git folders:

 [ankur@ankur ~]$ cd Documents/work/repos/others/fedora-packages/ [ankur@ankur fedora-packages]$ ls aeskulap dcm4che-test fpaste hiran-perizia-fonts libmpd memaker OpenNL python-hl7 subversion trash-cli cmake detex freemedforms Inventor libtorrent mpdas OSGi-bundle-ant-task rssdler suitesparse vtk comps django-authopenid gdcm klt libtpcimgio nifticlib pyclutter rtorrent toothchart xmedcon curlpp evolution-rss gnumed libgexiv2 libtpcmisc oldstandard-sfd-fonts pystache scout transmission zlib [ankur@ankur fedora-packages]$ for i in `ls`; do cd "$i"; git pull; cd ..; done remote: Counting objects: 5, done. remote: Compressing objects: 100% (3/3), done. Unpacking objects: 100% (3/3), done. remote: Total 3 (delta 2), reused 0 (delta 0) From ssh://pkgs.fedoraproject.org/aeskulap * [new branch] f17 -> origin/f17 ca0fa88..ef4f943 master -> origin/master Updating ca0fa88..ef4f943 Fast-forward aeskulap.spec | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) remote: Counting objects: 20, done. remote: Compressing objects: 100% (17/17), done…

View original post 12 more words

Ouch! My Bad

Subhodip's Blog

JavaScript is powerful and Backbone.js is a nice small utility that makes it even more powerful. I was playing with Backbone and Open layers thrown over a Django system.
However there is something weird that I faced and took a good amount of my time to figure out the problem.
This blog post is kind of self note so that I notice this the first place when I face such problem again.

What was I trying to do : Write a small backbone code to display Open layers(OSM) Map.
How was I trying to do :

(function($) {
var Map = Backbone.Model.extend({});

var MapView = Backbone.View.extend({

initialize: function() {
_.bindAll(this, 'initMap');
this.initMap();
},

initMap: function() {
// Initialize Basic Openlayers;
var center = new OpenLayers.LonLat(8110203.9998955, 2170000.4068373);
map = new OpenLayers.Map(this.el, {
projection: new OpenLayers.Projection("EPSG:900913"),
displayProjection: new OpenLayers.Projection("EPSG:4326")
});
var layers = [];
layers[0] = new OpenLayers.Layer.OSM(); //some more layer will…

View original post 183 more words

#Transifex now supports comments in Apple .strings i18n files

#Transifex now supports comments in Apple .strings i18n files. Only /* foo */ style comment in the line preceding the key value pair in the source file is saved as a comment for the key. The example below will explain this in a better way:

/*Comment for key1*/
"key1" = "value 1";

/* This comment will not be
included in key2*/

/* comment for
key2*/
"key2" = "value 2";

/* this comment will not be included in key3*/

"key3" = "value 3";

Well, I’m pretty sure that the above snippet explains which comments from source Apple .strings file are saved by Transifex. You can see the comment for a source string in its “Details” section in Lotte.

Comment for a source string imported from a source Apple .strings file

Start testing Transifex

 

How do you setup Transifex?

Here is all you need to know to setup Transifex: http://help.transifex.net/server/install.html

http://fosswithme.wordpress.com/2011/10/20/setup-transifex-in-virtualenv/ is another good write-up on how to setup and run Transifex in virtualenv. So, I’d be building on top of that to show you how to start testing Transifex using django-nose.

What packages will you need?

django-nose, nose, nose-exclude, coverage

You can install the above packages using

pip install <package_name>

Configure Transifex settings to enable django-nose Test Runner

cd <transifex source code's root directory>

cd transifex/settings

cp  90-local.conf.sample 90-local.conf

open and edit 90-local.conf and add the following lines:

INSTALLED_APPS += [

‘django_nose’,

]

TEST_RUNNER = ‘django_nose.NoseTestSuiteRunner’

cd ..

Now, save the file. Now you are good to go.

Start testing

python manage.py test <app_name>

for example:

python manage.py test resources

You can also test a particular test class like below:

python manage.py test resources.tests:TestJavaProperties

You can even run a particular test method:

python manage.py test resources.tests:TestJavaProperties.test_properties_parser

Run coverage on Transifex tests

django-nose has a plugin for coverage. So, you can run the above tests and collect coverage data.

For example:

python manage.py test resources.tests:TestJavaProperties.test_properties_parser --with-coverage --cover-package=resources.tests.formats

All the coverage results are saved in a .coverge file by default in the current directory. Although, running tests with coverage plugin of django-nose shows the coverage results by default. You can also see the coverage results in the usual way:

coverage -rm

You can also use grep along with the above command to filter the results displayed.

Well, that’s all you need to know to start testing Transifex.

What’s next?

Start testing transifex. If you find a test fails, try to find the reason why it failed. Read the traceback info properly. Find where the error took place. There are various reasons why a test may fail:

  • Test is not updated according to updates in code
  • Bug in code
  • A wrong test case

and others…

You can report the issues or any bug you find at http://trac.transifex.org/newticket. Feel free to submit a patch that fixes the issue. The patch will be reviewed by the Transifex upstream and if it is ok, it will be merged with Transifex’s code at http://code.indifex.com/transifex/.

Keep hacking 🙂