[GSOC] Easy Plugins Website

[Bhaskar Kandiyal]

Hi Everyone :)

My project for GSOC was to create a website to facilitate easy
downloading of Pidgin plugins where developers can publish their plugins
and users would easily be able to download them and install them through
Pidgin. I also had to improve the plugins window and add functionality
to automatically install and update the plugins.

Here's a brief summary of the work that I've done this summer:

I started my work by implementing the plugins window design that was
made by Tomasz Wasilczyk. It's not exactly the same since I moved some
UI elements to the WebKit webview (Preferences and other buttons etc.)
to save some screen space and to add a Category / Filters tree.

For the plugins website, instead of using Mozilla's Zamboni like I
initially planned to, I decided to create my own lightweight Django
project which incorporates some third party apps without the unneeded
features of Zamboni. I also incorporated Jinja2 in the website which is
a template engine that is alot better in terms of features and speed
than the default template engine of Django.

The website backend was the most challenging part of the project since I
had to add support for plugins that run on different platforms and
architectures. When developing the website I came across some
limitations of Django, mainly that it doesn't allow low-level control of
forms and form fields, because of this, designing forms for plugin
uploads posed quite a bit of challenge and took a lot of time to get it
right. Once I figured out how I could have some low-level control of
Django forms I added dynamic forms to the website so that the platforms
and architectures that the user has to select are not hard-coded (Thanks
to Eion for pointing this out :)).

Once the uploads were working the next thing was to decide how to serve
those uploaded files and for this I used two approaches:

1. Django SendFile - It uses mod_wsgi or X-SendFile to serve files from
a dedicated web server and can also serve files itself but that is slow
since Django is not meant to serve large static files.

2. Direct downloads - This approach uses a redirection to redirect to
the plugin file served by a dedicated web server. This needs some
configuration of the web server (like adding a Content-Disposition with
a filename HTTP header).

Either of them can be selected from the settings module.

Once the majority of the core features were implemented in the website
backend I started working on small features like the comments system,
user profiles, announcements, ratings and authentication. I got the
OpenID authentication working by using the Django-AllAuth app which made
things a whole lot easier. The users are now able to sign in using their
OpenID accounts and or Facebook and Google accounts (these are just the
ones I've enabled by default, LinkedIn, Twitter and a plethora of others
are also supported).

This is all about the backend, for the frontend I initially used
Bootstrap v2 for styling but then Bootstrap v3 was released so I
migrated over to it, this required some work but it was worth it as the
website looks much better with Bootstrap v3 and a custom CSS theme than
the previous one. Apart from the looks I also added some custom
JavaScript code mainly to the forms so that they are easier to work
with. The plugin upload form was initially a large form that could be
overwhelming to users so I split it up into a 'wizard' like format.

Now, coming to the plugins window, I improved upon the design by Tomasz
and since Ankit Vani implemented categories within plugins in his GSOC
project, I added a Categories / Filters tree to the window so that users
can easily filter plugins by categories or their state (i.e Enabled or
Disabled). As in the initial design a tab was also added with a WebView
that would open the plugins website so that users could install new
plugins easily.

Once I had the installation of plugins from the plugins window working I
went on to implement automatic updation. The updates are checked on a
weekly basis by using a custom API that I designed for the website using
REST and JSON. The updates can be interactive (Ask users which updates
to install) and non-interactive (Automatically install updates) based on
what the user chooses in the preferences. They are downloaded to an
'updates' folder first and installed on the next startup (so as to not
interfere with plugins that are enabled). Thanks to Mark Doliner for
suggestions on how to implement the timers for automatic updation and to
Jorge Villaseñor for suggesting that users should be notified before
installing updates :)

As for the challenges that I faced, the main challenge was developing a
website that supported both Python2 and Python3 and since Python3 is not
backwards compatible and not a lot of third-party libraries are
available for Django that support Python3, I had to implement a lot of
the things myself. I also had to port Django SendFile to Python3 (with
backwards compatibility) so that it would work well with both the
environments.

So, that's it, that's my project summarized (that was rather long :)). I
can't include everything as that would be even longer but there's a lot
more to the project than just the above summary :D

The current status of the project is that it's in a working state
(Although needs extensive real-world testing) and what I had planned to
implement when I had started working on the project has pretty much been
implemented. But still there's a lot to do and improve and I would love
to keep on developing / improving / maintaining this project even after
GSOC.

I would like to thank everyone who helped me out in testing and giving
suggestions especially Mark Doliner (For helping out with my repository
problem and giving awesome suggestions :)), Tomasz Wasilczyk (For the
plugins window design), Ankit Vani (For testing the website and
suggestions for the plugin category filters), Jurre Van Burghen (For the
excellent suggestions when I started working on the project), Robert
Vehse (For finding out the CSS bug :)) and Jorge Villaseñor (For the
update notification suggestion).

And last but certainly not the least special thanks to my mentor Eion
Robb for testing out the project, giving suggestions, answering my
questions (Apologies for bugging you at awkward timings :)) and
supporting me throughout the course of the project. It was awesome
working on this project with you!

It was a great experience working on the project and I would love to
keep working on it. For those who wish to checkout the project
repositories please follow [1] and [2].

Cheers,
Bhaskar Kandiyal

[1] - http://hg.pidgin.im/soc/2013/bhaskar/plugins-website/
[2] - http://hg.pidgin.im/soc/2013/bhaskar/plugins-window/