Skip to content
Sep 14 / PK Shiu

Reference Django Admin Site URLs Symbolically

Sometimes We want to expose part of  the Django admin site to the user. Django provides a way to build those admin site page URLs symbolically using the built in URL reversing system, using the reverse function. This feature is well documented, but sometimes hard to find. The correct url is here, on reversing admin URLs.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit
Aug 29 / PK Shiu

mkvirtualenv problem on Mac OS X Lion?

I finally get around to setting up a django development environment on my MacBook Pro running Lion. I immediately run ino the error “Could not call install_name_tool” error when I try to run mkvirtualenv

The solution is very simple (and silly). We always remember to install XCode before setting things up. But by downloading and “installing” XCode from the AppStore only download and install the XCode Installer. You have to run the downloaded installer to get XCode install.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit
Mar 27 / PK Shiu

Should I Upgrade to Django 1.3?

Should I upgrade my Django application to the current version? The very short answer is Yes.

The slightly longer answer is “Yes if your application is more than a static site used by more than 10 people”.

Starts with the general first. Upgrading an application that is already in production is always a hard decision. What are general pros and cons?

Cons

  1. Cost — time and money. Someone has to (be paid) to do the work.
  2. Break something — for a non trivial update, chances are you have to make some code changes. When you touch the code, you may break something. To make it worst, what if the original developers are gone?
  3. Dependencies — if your application uses other third party software, have they upgraded? If they are incompatible with the new version, it will break.

Pros

  1. New Features — The availability of a new feature on it’s own is not a good enough reason for upgrading. “The site works now, why change?” A developer may want to try out a new feature on a new release. But it has to have real benefit to the application to make this a Pro instead of a Con.
  2. Speed Improvemence — platform vendors often upgrade a platform to improve performance. This is often a legitimate reason for upgrading.
  3. Avoid obsolescence — This point applies especially to web frameworks. The web environment changes quickly. Platform provider may choose to end support for older versions quickly.
  4. Security — This point also applies especially to web frameworks. New attacks are being develop every day. Beyond small emergency security fixes, a framework may add new security feature that is useful.

Let’s focus on Django. Speaking from real world experience, we have upgraded production django sites from 1.1 to 1.2 to 1.3 each time a new release comes up. Our reasons for upgrading are driven primarily by new features that we can use. Our sites are all transactional in nature. The more sophisticated the site, the more of a reason to upgrade.

New Features We Like: Working backwards from the current 1.3 release:

Django 1.3 Features that we like

  • Logging — finally we don’t have to use our own logging module anymore
  • ClearableFileInput widget so that standard forms can easily delete previously uploaded file content without crazy hacked solutions
  • Support for lookups spanning relations in Django admin’s list_filter option — this makes list_filter actually useful. Often with a large object hierarchy it only make sense to use list_filter to limit/segment the data at the top levels of the hierarchy.
  • Class Based Views — This is a major new feature. I will post more on this later.
  • Unittest2 — The new unittest2 test support is now provided within Django 1.3 even if you are running an older release of python. Many features in unitest2 helps you write tests faster.

How to Convince Your Clients or Management to Upgrade?

“If it ain’t broke don’t fixed it” maybe the response that you are getting. The best approach for us is to roll the upgrade work in with the next release. Initial conversion should not take more than one or two days. If you have a one week or a two weeks software release coming up, add the conversion work to it. That is what we do. And in the mean time, try upgrading on smaller non production sites.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit
Mar 24 / PK Shiu

Deprecation warning with get_db_prep_value for Django 1.3

If you are getting this warning:

DeprecationWarning: A Field class whose get_db_prep_lookup method hasn't been updated
to take `connection` and `prepared` arguments.

Are you using django-picklefield ? If so, upgrade to the latest 0.1.9 version.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit
Jan 3 / PK Shiu

Django database migration tool: south, explained

Introduction

If you are using Django for production level application, you will need to use south. Requirement changes, and therefore your data model will change over time. South is a great tool, but it is complicated. You do not want to make a mistake when migrating an application in production. This is a detail look at how it interacts with your application so that you can understand and use it better.

Interaction

South interacts (reads and writes) with four different items in an application. This is the most confusing part to me. The four items are:

  1. models.py — South reads this to determine your current data model.
  2. migrations/*.py — South creates a sub directory inside your app, and creates for you a migration file for each database migration generation. You can also create these by hand, but normally you will let south creates them for you.
  3. south_migrationhistory table in your database — when you install south, it creates it’s own table and use it to maintain states. Specifically it records the state of the database in this table. South assumes the application schema in the database is consistent with what it records in this table.
  4. your application’s schema in your database — south create and update the schema for you according to the database migration generation, which is the ultimate purpose of using south.

Different commands in south interacts with these items differently:

Normal Migration

Let is start with a normal database schema migration, from generation N to generation N+1. There are three steps in the migration:

1. The developer (you) change the models.py file,
   updating the application's data model.
2. Run manage.py schemamigration app_name --auto to create
   a migration file for generation N+1.
3. Run manage.py migrate app_name to update the database
   schema and migrationhistory table to generation N+1.

This diagram shows what are the inputs and outputs to each step. (Note, in step 2 it reads all of the migration files from all previous generations):

Initial Migration

Let’s add the very first step when using south on a new applicatoin. The initial migration obviously does not have any previous south information. So there are still three steps, but the argument passed to schemamigrate is a little different:

1. The developer (you) creates the first models.py file,
   defining the  application's data model.
2. Run manage.py schemamigration app_name --initial to
   create a migration  file for generation 1.
3. Run manage.py migrate app_name to create the database
   schema and  migrationhistory table to generation 1.

This diagram shows what are the inputs and outputs to each step, adding to the previous diagram:

Converting an Application

Converting an application is a little different because the database schema and the models are already in sync. Somehow we need to “trick” south into creating the other two items, the migration file and the migrationhistory table entry. We also need to deal with the conversion on the first instance vs other instances differently. For the first instance you will create the migration file, and for all other instances you will use the migration file to simply create the migration history table entry.

On the first app instance:

1. Run manage.py convert_to_south app_name to create the migration
   file for generation 1, and also to create the migrationhistory entry.

On other app instances:

1. Run manage.py migrate app_name 0001 --fake to create
   the migrationhistory without changing the database schema.

Diagram:

Testing and Trying Migration before

There are several different things you can do to “dry run” your migrations.

Schema Migration Generation Test Run

./manage.py schemamigration my_app_name --auto --stdout

The -stdout argument will have South generate and print out the migration code on the console instead of writing it out to the migrations directory. You can do this before you do the actual database migration schema generation. Note that schemamigration only creates the next migration script in the migrations directory. Worst case is you can just delete that file if you decided the migration schema is not needed.

Migration Dry Run

The migration operation can also be tested before actually running to migrate a database:

 ./manage.py migrate myapp --db-dry-run

This will run the migration except the database is not changed, and the migration is not recorded in the history table in the database.

Useful Commands

Another useful command is the list command. It shows all the migrations defined in the migration files, and whether they have been applied (by reading the migrationhistory table entries):

manage.py migrate --list

Deleting Migrations

Sometime I need to reset a migrated environment. This is particularly useful during initial development of an app. This special command, migrating to the “zero” state, will remove all migration history in the database, but leaving any migration files intact.

manage.py migrate my_app_name zero

Bonus

This is my initial sketch for my explanation diagrams:

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit
Oct 7 / PK Shiu

Fix Python source code to use spaces instead of tabs

What if someone gave you a Python source file that is indented using tabs? If you are using emacs, the following commands will let you convert it back to using spaces:

# first set the buffer tab width to 4 (or whatever you like)
M-x set-variable <return> tab-width <return> 4

# then mark the entire file
C-x h

# do untabify to convert:
M-x untabify <return>

That’s it!

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit
Oct 6 / PK Shiu

Django production error handling

Django provide a nice mechanism to report error in its “batteries included” goodness. You can easily setup the Django environment so that it will send you an email when a “server error” occurs. You just need to make sure the following is setup:

Outbound email working

The django environment must be able to send outbound emails. The actual requirement depends on your server environment, but you definitely need to have correct values setup for:


settings.EMAIL_HOST
settings.EMAIL_PORT
settings.EMAIL_HOST_USER
settings.EMAIL_HOST_PASSWORD

Admin users

settings.ADMINS -- this is a list of lists (or more accurately tuple of tuples)
settings.SERVER_EMAIL -- email address of the error reporting from address

Debug Setup

settings.DEBUG=False

500.html and 404.html

Once DEBUG is off, Django will want to display your 500 or 404 page. Create these pages and make them available on one of our template directories.

Example

Here are some sample entries from my settings file:


EMAIL_HOST='smtp.webfaction.com'
EMAIL_PORT=25
EMAIL_HOST_USER='my_mailbox_name'
EMAIL_HOST_PASSWORD='my_mailbox_password'
SERVER_EMAIL='webmaster@imperial-consulting.com'
ADMINS=( ('PK Shiu', 'support@imperial-consulting.com'),)
DEBUG=False

Reference

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit
Sep 5 / PK Shiu

Django template: putting commas between a list of items

This is a very common usage pattern. You have a list of items for display, perhaps a list of tags. The list can have zero, one or more items. How do you, using the Django template language, put commas between each item?

1. This simple way will not produce good result for a list of one item:


toppings = [ 'cheese','tomatos','pineapple' ]
or toppings = ['cheese']

{% for t in toppings %}
{{ t }} ,
{% endfor %}

that will output:
cheese, tomatos, pineapple,

or

cheese,

Note the ugly trailing comma.

2. This is the smart way using the template variables available in loops:


{% for t in toppings %}
{% if not forloop.first %}, {% endif %}
{{ t }}
{% endfor %}

that will output:
cheese, tomatos, pineapple

or

cheese
No more trailing commas, thanks to the built in forloop variables.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit
Aug 6 / PK Shiu

Moving from Java to Django

I gave a short presentation on Django to the Cambridge Python Users group earlier. Nate has a great writeup of the event and the other presentations that evening. I just want to share the slides here. The slides are just visual reminders and do not stand on their own. If you want more info free feel to shoot me an email.

I switched from J2EE to Django as my sole web application platform two years ago and has not looked back since. It allows me to develop, and more importantly maintain, web apps faster and better. It is more time and cost effective for my customers and I.


Slides from J2EE to Django Presentations at Cambrdge Python Group from PK Shiu on Vimeo.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Twitter
  • Google Bookmarks
  • Add to favorites
  • email
  • FriendFeed
  • Google Buzz
  • LinkedIn
  • Tumblr
  • Reddit