Use Git Hook to Automate the Deployment of Code to Staging or Production Servers

Continuous integration is an integral part of software development today. Git hooks help us accomplish continuous integration, and I can’t imagine how difficult it would have been without git hooks.

If you use git for development then following can help you setup automated deployment of code to your staging and production servers. I used the following to setup staging environment for one of my Django projects, but it can be used with other type of web applications as well.

I wanted to be able to run the app as it will run in production, so I wanted to use Apache with WSGI to host the application. So when I push the code I wanted following to happen in order:

  1. Update the application code hosted on server.
  2. Restart apache.

Staging or Production Server Setup

You need to host a bare repository on your server. If you already have that you can skip this section, below I will list down the steps to replicate how my server is setup and how git bare repository is hosted.

  1. Create a user to host git repository.
    useradd -m git
  2. Set up the password for new user or if you use key based ssh logins set it up for new user and make sure you are able login to remote machine from your development machine.
  3. Login as git on your server and clone your git repository.
    git clone --bare

    If you run the above command in home directory of git user you will have a new directory created named “<RepositoryName>.git”. Git will automatically add ‘.git’ at the end of bare repository to differentiate them from the non-bare repositories.

  4. Goto your development machine and setup new git remote, I am using ‘staging’ in the following command but you can name it what ever you want.
    git remote add staging git@server.com:.git
  5. Do a test push to verify everything is working and you can successfully push to your repository on staging.
    git push staging +master:refs/heads/master

Setup Git Hooks For Automated Deployment

If you followed the steps above then your bare git repo will be at “/home/git/<RepositoryName>.git”. But for simplicity we will use following variable names.

Bare Repository Path on Remote Server: <RepositoryPath>
Application deployment path (document root): <AppPath>

  1. Goto your <RepositoryPath>/hooks
  2. Create a script named: post-receive
    Note: It can be a executable script in any language, but I prefer bash for such trivial tasks.
    Note 2: Don’t forget to change the variables in the script as per your setup.

    #!/bin/bash
    while read oldrev newrev ref
    do
        if [[ $ref =~ .*/master$ ]];
        then
            echo "Master ref received.  Deploying master branch to production..."
            git --work-tree= \
                    --git-dir= checkout -f
            echo "Restarting Aapche"
            sudo /usr/sbin/apache2ctl restart
        else
            echo "Ref $ref successfully received."
            echo " Doing nothing: only the master \
    branch may be deployed on this server."
        fi
    done
  3. Make script executable.
    chmod +x post-receive
  4. Do a git push and you should see output like. Notice the lines starting with “remote:”. These are the outputs from our script.
    Counting objects: 4, done.
    Delta compression using up to 8 threads.
    Compressing objects: 100% (4/4), done.
    Writing objects: 100% (4/4), 375 bytes | 0 bytes/s, done.
    Total 4 (delta 3), reused 0 (delta 0)
    remote: Master ref received. Deploying master branch to production...
    remote: Project update successful... Restarting Aapche
    To git@.com:.git
       50244ea..e46a9f0 master -&gt; master

Django Specific Post-receive script

For deployment of Django app there are some extra django specific commands which you might want to run before restarting your apache, so I modified the script that I am using to do following as well before actually restarting apache, and the apache is only restarted if all the commands are successfully completed.

  1. export Python virtual environment path.
  2. source virtualenvwrapper.sh
  3. load the virtual environment
  4. make sure pip-tools is installed
  5. run pip sync to make sure all project dependencies are installed and upto date
  6. run db migrations
  7. collect static files
  8. django check to verify everything is good
  9. restart apache

Below is the script as it is.

#!/bin/bash
while read oldrev newrev ref
do
    if [[ $ref =~ .*/master$ ]];
    then
        echo "Master ref received.  Deploying master branch to production..."
        git --work-tree=/var/www/MySite \
		--git-dir=/home/git/MySite.git checkout -f
	cat &lt;

Develop Your First Android Application in Python

Android is getting popular among mobile users very rapidly, and the fact that Android’s user base is increasing the demand for more usable applications on Android is increasing too. The demand for more applications is causing more and more developers to get into Android development, unfortunately until now the only way to develop for Android was to develop in Java.

But thanks to Google that one of their employee has come up with a solution that allows not only Python programmers but Perl, lua and bash programmers to develop on Android. The new project that has come out from Google is called SL4A and it allows you to use Python to develop for Android. You can get to know more about SL4A from the home page of the project on Google Code. This article tells you the exact steps you can take to setup Android SDK, Install SL4a and Python for Android and then Write your first Android Application in Python and run it.

Setup Android SDK and Android Virtual Device.

First you will have to download latest Android SDK from Google and setup your Android Virtual Device. If you like you can directly develop your Android application on your phone device. Skip to “Installing SL4A and Python for Android” if you want to directly develop on your phone.

1. Download Android SDK. For Linux download the tar ball and for Windows download the executable (recommended).

2. Install the SDK. You might also want to read the Installing the SDK document on Android’s developer site.

On Linux:

Extract the tarball in any directory of your choice.

On Windows:

Run the executable to install the Android SDK.

3. Go into the “tools” directory inside the extracted or installed directory.

4. Run the “android” executable.

On Linux:

$ ./android

On Windows:

Double click the executable to run it.

5. The android executable will start the Android SDK and AVDManager.

AVD (Android Virtual Device) manager

6. Click on the Create New AVD to setup the new AVD for your Android application development.

Create new AVD

7. When the new AVD is created, select it and click on Start. You will see a Android device emulator screen and an Android Logo, the Android OS is booting, you will have to wait for some time. After some time you will get a fully loaded Android OS on your Android emulator to develop and test your Android Application.

Android Emulator

Installing SL4A and Python for Android

Scripting Layer For Android is the main Android Application you need to be able to run your Python applications on Android, so first you need to install SL4A on either your Android emulator or on the real Android device in case if you are going to do the development directly on your device. Follow the steps below to install the SL4A on your Android.

1. Open Android Browser and browse to the SL4A home page: http://code.google.com/p/android-scripting/.

2. Tap on the QR code icon (the big square box) to download the SL4A application.

Android Browser SL4A home page

3. Tap on the download notification to install the app.

4. Once the installation is complete tap OK and then go back to SL4A home page on your Android Browser.

5. Tap on Python for Android in the Featured Downloads section to download Python for Android.

SL4a Python for Android Download

6. Tap on the download notification to install Python for Android.

7. Once the Installation is complete go to SL4A application which is located under your applications menu. You will see some python scripts that got Installed along with Python for Android.

SL4A Python for Android Test

8. Tap the test.py, a menu will popup.

SL4A run menu

9. Tap the second icon from left. The test.py script will execute and will demonstrate some of the possibilities of the SL4A API available for your Python programs.

Note: The most left icon is to run the application in terminal, the second from left is to run the application normally, the third from left is to edit the script, the fourth from left to rename the script and the last one from left is to delete the script.

Writing the First Android Application in Python

Now that our Android is ready and you have already tested that everything is working fine, its time to write your first Python application. You will write a very simplest version of hello world, this application will ask your name and then it will greet you. Thats it.

1. Copy the following code to greet_app.py on your computer.

import android
 
app = android.Android()
 
name = app.dialogGetInput("Enter your name:", "Please enter your name: ").result
app.dialogDismiss()
 
app.dialogCreateAlert("Greetings!!!", "Hello %s, how are you?" % name)
app.dialogSetPositiveButtonText('OK')
app.dialogShow()
app.dialogGetResponse().result
app.dialogDismiss()

2. Goto “platform-tools” directory which is found under the Android SDK directory.

3. Copy (push) the greet_app.py to your Android by running the following command.

$ ./adb push /sdcard/sl4a/scripts/
4. Press the menu button on your Android device or Android emulator, and tap Refresh.

5. greet_app.py will appear in the script listing, tap the greet_app.py script.

6. The SL4A run menu will appear, click the second icon from left.

SL4A First Python Android Application

7. A popup will appear asking you for your name, enter your name and click OK.

SL4a first Python Android Application running

8. Another popup will appear with a greeting message for you.

SL4A Greetings

 

Congratulations!!! You are done here. Its now up to you how you make use of the Android API available via SL4A. Best of luck.

You should subscribe to the feed to stay updated about the next article that will describe how you can build a stand alone .APK application from your Python script.

The Zen of Python

The famous Zen of python was written by Tim Peters. As now I will be writing posts here about Python too, so I thought I should start with the Zen of Python.

Small Easter Egg

If you have Python Interpreter you might want to try out the following.
import this

The Zen of Python

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.
Now is better than never.
Although never is often better than right now.
If the implementation is hard to explain, it’s a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea — let’s do more of those!

Stay tuned for the some real python stuff on your way…

How to use oauth-python-twitter

Using OAuth for authentication is quite different then using Basic Auth in many ways, first major difference is that one don’t know the username until it is requested from twitter after completing the authentication process.

Details

You will need following information to start.

  • Consumer Key
  • Consumer Secret

If you do not have above, you can request them from here: http://twitter.com/oauth_clients

The following are the steps that should be performed to authenticate user and get user information from twitter, the rest is same as using python-twitter.

  • Get the Request Token from twitter
  • Get Authorization URL
  • Get the Access Token from twitter
  • Get user information

Get the Request Token from twitter

twitter = OAuthApi(CONSUMER_KEY, CONSUMER_SECRET)
 
request_token = twitter.getRequestToken()

Get Authorization URL

using same request_token from previous step

authorization_url = twitter.getAuthorizationURL(request_token)

Now send the user to authorization URL, for allowing access to the application.

Get the Access Token from twitter

Once the user return from twitter, we need to request the access_token from twitter for futher aunthenticated api calls on behalf of user.

NOTE: Need to create the new instance of OAuthApi using request_token from first step.

twitter = OAuthApi(CONSUMER_KEY, CONSUMER_SECRET, request_token)
 
access_token = twitter.getAccessToken()

Get user information

This api call was not present in python-twitter and the user info call expected that there we know the username before making call, but this is not the case with OAuth, so we have to request User Information so we can use it in our application.

NOTE: Need to create the new instance of OAuthApi using access_token from last step.

twitter = OAuthApi(CONSUMER_KEY, CONSUMER_SECRET, access_token)
 
user = twitter.GetUserInfo()

Now use the rest of python-twitter api calls as you used to use them with basic authentication, for more information see the wiki at:http://code.google.com/p/python-twitter

OAuthApi, an OAuth Extension to python-twitter

If you are a python developer and you tried to develop some application for twitter, you must have came acrossed python-twitter. Which provides you a pythonic interface to twitter, by allowing you to access almost all twitter objects as python objects. python-twitter is still in devleopment, and hence still does not have the OAuth compatibility. I am being a oauth pro, deicided to write a derived class from python-twitter, to allow OAuth to be used as the authentication mechnism for python-twitter.

python-twitter is being developed by our beloved google’s employee: DeWitt Clinton

You can get the python-twitter at: python-twitter

You can get the oauth extension to python-twitter at: oauth-python-twitter

In the next following posts I will posting about how I am using it with Django specifically and how I have writter my TwitterAuthBackend for Django. The whole purpose of publishing the code is to get improvements and reviews on the code, so I have allowed non-members to review the above project on code.google.com, you are more then welcome to post your coments here or review the code on the project. Let me know if you don’t understand anything by asking questions in comments.

P.S I have used the same license as was used for python-twitter.

Happy twittering and developing 🙂

Google App Engine: Do not let your engine stop

As you all might have noticed that I was not posting for a while now, actually had been taken overy by few projects. But today I thought to post about the updates from Google App Engine. Google App Engine have released the Cookbook where you can submit your recipies or can benefit from other people’s snippets of code.

One more thing that Google App Engine team has done which is really useful, they have included the documentation with SDK, so if you are offline you will still have access to SDK. This will not let your engine stops even if you are not connected to internet.

Google App Engine is getting popularity.

If you haven’t been following Google App Engine Blog, according to latest post the Google App Engine is getting popularity and many programmers are learning python just to play with App Engine. According to Marzia Niccolai on Google App Engine Blog:

I’ve returned from the App Engine hack-a-thon in Chicago a Superfan of the App Engine community. We had lots of fun meeting App Engine developers, but why let me tell you about it? A couple of attendees wrote great summaries of the day: A Ruby on Rails developer giving Python a whirl, and a developer who worked on testing with App Engine. We saw people working on iPhone apps, OpenSocial and App Engine, and even a chat application!

So, if you want to host your application on reliable Google platform then you gotta learn Python to make your next app in Google App Engine. Google App Engine is going to be the next big thing if they started supporting rest of the languages like PHP, Ruby and Perl which they have already promised. So stay tuned.

Rapid Development with Python, Django, and Google App Engine

Google I/O the developer event that happened in SanFransisco, many were able to join it but there are thousands of people who were not able to join it Google has not forgotten them. The videos of almost all the talks has been posted but the best one is “Rapid Development wiht Python, Django and Google App Engine“.

Its a must watch video so I decided to share it here. I hope you will benefit from it. You can get the presentation slides from here.

If you like the above video then there are few other good videos too related to Google App Engine, so if you ware interested in learning Google App Engine and not only that but there are videos for other Google technologies which include Gears, Gadgets, OpenSocial and gData. Have look at Google I/O session videos posted with slides

Now YOU can sign up for Google App Engine

Wow! Great news for python developer out there, Google has open the signup for their App Engine. Now every one can signup for the App Engine account. If you haven’t got your account yet and want to be on the bleeding edge of web development, then go and signup for App Engine and have some fun with App Engine SDK then you can upload you app.

Google App Engine has also introduced to new API which includes Image manipulation and our all time favourite memcache API. So, its time to put the fun back into web development and lets develop some useful apps based on App Engine.

If you are Django Developer then you can put your Django Apps on Google App Engine too. Happy developing.

Note: If you are not a developer and want to see something developed on App Engine, then leave a comment and may be some of us will do it for fun.