Pycon 09 concluded recently in Chicago. Although I could not attend the conference, I have no doubt in my mind that it is one of the most well managed professional conferences.
The conference videos are up on blip.tv. I went through some of them. The audio quality of the recording has been extremely good. One of the best live recordings I have seen so far. The videos are shared freely under the creative commons license so that more people in the community can benefit from them.
I will surely try to make it to the conference next year.
Tuesday, April 21, 2009
Saturday, April 18, 2009
TurboGears2 Geospatial Framework - GSoC 2009 Project Proposal
Summary
The tgext.geo [1] project was started as a part of SoC 2008. It provides basic GIS functionality to TurboGears2. It uses MapFish server to provide a RestFul web interface for geospatial data with a PostGIS backend. it also integrates Tilecahce for caching and serving of map tiles. It is proposed to enhance tgext.geo project into a Geospatial Web Framework using TurboGears2. This would involve developing an SQLAlchemy GIS Extension to support various spatial database features. In addition FeatureServer [2], a popular python GIS server will be integrated with TG2 to provide a much richer set of GIS functionality.
Description
The tgext.geo project was developed as part of GSoC 2008 Toscawidgets project. It makes use of Mapfish server to provide a RestFul web interface to GIS data stored in a PostGIS spatial database. It also uses Tilecache to cache and serve map tiles making TG2 a fast map tiles server platform. While tgext.geo, in its present form, could address a large variety of geographic applications, there are areas where tgext.geo needs to improve to in order to make TG2 a true geospatial web framework.
Geodgango extends the django ORM to support geospatial data objects. ATM SQLAlchemy, the ORM component of TG2 stack, does not support geospatial databases like PostGIS, SQLiteSpatial or MysqlSpatial. Mapfish uses a module mapfish.sqlalchemygeom to extend SA to support PostGIS. It is proposed to extend SQLAlchemy to add support for these spatial databases.This will be achieved by using custom types feature of SA to support geospatial datatypes.
FeatureServer is a python geographic feature server. It provides a large number of GIS services types like WFS, GML, GeoJSON, GeoRSS, etc. Featureserver could be integrated with TG2 as a best of breed GIS WSGI app. FeatureServer uses the FeatureServer.DataSource module for storing geospatial objects. The various supported DataSources are PostGIS, SQLite, WFS, OGR (Shape File), etc. A DataSource for SQLAlchemy GIS Extension could be added so that the backend database could be integrated with the TG2 ORM layer. By integrating FeatureServer with the TG2 we could have a Geospatial Web Framework with all the goodies provided by TG2 like auth & auth, transactions, multiple databases, toscawidgets and indeed all the other wsgi middleware components. While integrating FeatureServer as a WSGI app would be simple, implementing the new SA based DataSource would require some work.
Testing
Care will be taken to have 100% unittest coverage of the SA GIS Extension. TG2 Featureserver integration would also be tested using WebTests based functional tests.
Documentation
Documentation would be delivered as tutorials and api docs. Documentation on getting up and running with gis infrastructure (e.g. setting up postgis, getting sample gis data, etc.) would be delivered. Sphinx will be used for generating the docs.
Milestones
As per GSoC timeline [3] the coding period begins on May 23 and ends on Aug 16. There are a total of 13 weeks. The important milestones towards completion of the project are proposed as follows:-
Limitations
[1] http://code.google.com/p/tgtools
[2] http://featureserver.org/
[3] http://socghop.appspot.com/document/show/program/google/gsoc2009/timeline
The tgext.geo [1] project was started as a part of SoC 2008. It provides basic GIS functionality to TurboGears2. It uses MapFish server to provide a RestFul web interface for geospatial data with a PostGIS backend. it also integrates Tilecahce for caching and serving of map tiles. It is proposed to enhance tgext.geo project into a Geospatial Web Framework using TurboGears2. This would involve developing an SQLAlchemy GIS Extension to support various spatial database features. In addition FeatureServer [2], a popular python GIS server will be integrated with TG2 to provide a much richer set of GIS functionality.
Description
The tgext.geo project was developed as part of GSoC 2008 Toscawidgets project. It makes use of Mapfish server to provide a RestFul web interface to GIS data stored in a PostGIS spatial database. It also uses Tilecache to cache and serve map tiles making TG2 a fast map tiles server platform. While tgext.geo, in its present form, could address a large variety of geographic applications, there are areas where tgext.geo needs to improve to in order to make TG2 a true geospatial web framework.
Geodgango extends the django ORM to support geospatial data objects. ATM SQLAlchemy, the ORM component of TG2 stack, does not support geospatial databases like PostGIS, SQLiteSpatial or MysqlSpatial. Mapfish uses a module mapfish.sqlalchemygeom to extend SA to support PostGIS. It is proposed to extend SQLAlchemy to add support for these spatial databases.This will be achieved by using custom types feature of SA to support geospatial datatypes.
FeatureServer is a python geographic feature server. It provides a large number of GIS services types like WFS, GML, GeoJSON, GeoRSS, etc. Featureserver could be integrated with TG2 as a best of breed GIS WSGI app. FeatureServer uses the FeatureServer.DataSource module for storing geospatial objects. The various supported DataSources are PostGIS, SQLite, WFS, OGR (Shape File), etc. A DataSource for SQLAlchemy GIS Extension could be added so that the backend database could be integrated with the TG2 ORM layer. By integrating FeatureServer with the TG2 we could have a Geospatial Web Framework with all the goodies provided by TG2 like auth & auth, transactions, multiple databases, toscawidgets and indeed all the other wsgi middleware components. While integrating FeatureServer as a WSGI app would be simple, implementing the new SA based DataSource would require some work.
Testing
Care will be taken to have 100% unittest coverage of the SA GIS Extension. TG2 Featureserver integration would also be tested using WebTests based functional tests.
Documentation
Documentation would be delivered as tutorials and api docs. Documentation on getting up and running with gis infrastructure (e.g. setting up postgis, getting sample gis data, etc.) would be delivered. Sphinx will be used for generating the docs.
Milestones
As per GSoC timeline [3] the coding period begins on May 23 and ends on Aug 16. There are a total of 13 weeks. The important milestones towards completion of the project are proposed as follows:-
- Studying docs and looking around SA and FeatureServer codebase : W(-3) - W(-2)
- Writing experimental code discussing issues on ML IRC : W(-1) - W0
- Minimal working SA GIS Extension with PostGIS db : W1 - W2
- All tests for written code pass and code review : W3 - W4
- Add support for Mysql Spatial and SpatialLite : W5 - W6
- Tests for new code and code review : W7
- TG2 FeatureServer integration with tutorial and sample code : W8 - W9
- SA backend for FeatureServer : W10 - W11
- Tests and code review : W12
- Sphinx documentation and full TG2 GIS tutorial : W12 - W13
- Improve test coverage, Improve docs, Write report : W13
Limitations
- Exams during W0 and W1 - More time has been kept in W0 to W4 to make up for the loss
- May have some issues due to my timezone (UTC + 05:30) - Effort would be made to work more at night to synchronize time with West
[1] http://code.google.com/p/tgtools
[2] http://featureserver.org/
[3] http://socghop.appspot.com/document/show/program/google/gsoc2009/timeline
Friday, April 3, 2009
Sahana Project Proposal
1. Personal Details
Name: Sanjiv Singh
EMail: singhsanjivk@gmail.com
IRC: sanjiv
Education and Experience:
I am doing my Masters in Information Technology from Punjab Technical University, India. I already have a masters in Geophysics and I have been involved with programming for the last 6 years. I have programmed in projects of varying sizes using languages such as C, C++, Java, PHP, Perl and Python. I fell in love with python in 2006 when I started using it for writing sys admin scripts. I started using TurboGears web framework when I was looking for a framework for developing a web application in python. I have also had a look at web2py which is a project requirement for this project and I found it very similar to TurboGears. So I am confident that I will become productive on web2py within a few hours.
I am specially interested in open source development. Google Summer of Code 2008 gave me a chance to get associated with open source developers. I successfully completed the SoC project "Toscawidget Library Using jQuery and OpenLayers"[1] with the TurboGears team under the mentorship of Christopher Perkins. In addition to the proposed GSoC project I also developed a TurboGears Extension tgext.geo[2] that adds geographic capabilities to the TurboGears Web Framework. Since then I have been involved with the TurboGears community and have been contributing in several ways, e.g. contributing to TG subprojects (e.g. ToscaWidgets, tgtools, etc.), documentation, participating in sprints, etc. My Open Source development profile can be seen here . I have also had the opportunity to work for a midium sized Turbogears based ajaxified financial application which is already in production use. The front end for the app was developed using the ExtJS Javascript library which has given me a reasonable level of expertise in Javascript and the ExtJS framework too.
Why am I Suitable for the Project:
I have a reasonable level of expertise with most of skill requirements for the project, i.e. Python, Javascript, Geospatial technologies, OpenLayers and ExtJS as has been brought out above. So I will be ready to get started very quickly with a focus on deliverables rather than trying to learn about the technologies involved.I have a passion for Open Source development and could become a long term contributor to Sahana project.
What do I gain from the Project:
I feel the best way to spend the summer as a student is to participate in the summer of code program. Also an opportunity to work with the Sahana team would get me involved intimately with the project. As Sahana supports a humanitarian cause, this long term association would give me immense self satisfaction.
2. Personal Availability
- I have reviewed the Important Dates and Times (http://code.google.com/opensource/gsoc/2009/faqs.html#0_1_timeline_5354032302481437_) for GSOC2009
- I will have exams from May 18 to May 30. This means I will not be actively involved during the last week of community bonding period and the first week of coding period.
3. Project Abstract
The Geospatial Framework for Sahana would provide GIS capability to the Sahana project. The Sahana GIS project page lists several GIS applications that are planned for Sahana the most important one being victim tracing and tracking. The Geospatial Framework would provide a backend and frontend solution for storing, retrieving, handling and processing geospatial data in Sahana. The backend component will be closely integrated with web2py with an option for using GeoServer / FeatureServer / MapServer as decided by the Sahana core team. The frontend will be developed using OpenLayers or a combination of OpenLayers and ExtJS.
An appiclation like tracking and tracing would be best utilized using gps enabled phones / devices. So a facility to upload GPS tracks / positions will also be provided. Eventually it should develop into a phone / handheld based application wherein the location is updated directly from the phone handset.
4. Project Plan
Location awreness is an extremely important aspect of disaster management. Where are the victims and where are my resources? Answeres to these two questions will ensure effective and most optimal utilisation of limited resources towards disaster management. The Geospatial Framework for Sahana will address this problem and facilitate storage, processing, retreival and visualization of spatial (location) data in Sahana Disaster management System.
The technologies to be used for backend and frontend in the framework is not yet finalized. It is expected that these would be discussed on ML and finalized before the coding period begins. In case the backend is implemented in FeatureServer[3], which is python gis server, web2py controller will be able to create FeatureServer objects and save them to Feature Datasore, like shapefile (using OGR) or PostGIS, or SQlite. PostGIS supports advanced spatial capabilities. So switch from current mysql to postgresql needs to be discussed on ML. This would involve sahana modules for handling gis data. The modules will be called in the controller. In case the backend is implemented using GeoServer, features could be stored and accessed directly from client (openlayers) using WFS / WMS or through web2py using the python OWSLib[4] which implemnts OGC services (WFS, WMS) and could server-chain to GeoServer. In this case the sahana gis module will call the owslib services from the OWSLib package and read / write data from / to GeoServer using WFS / WMS (GetFeatureInfo).
The frontend will be implemented using the OpenLayers Javascript Mapping Toolkit.It would be interesting to use OpenLayers along with ExtJS framework which provides sophisticated widgets. The ExtJS layout container widgets could be used to hold the maps and tree widget to create the layer selection tree. Such toolkits are already being used in MapFish. There is a separate project launched under OSGeo recently called GeoExt. It uses OpenLayers and ExtJS. GeoExt could be evaluated and used if found suitable. The deliverables for the frontend would include a set of openlayers / extjs widgets for displaying the Map, controls for zooming and panning, controls for layer switching / layer tree and control for editing.
A facility for uploading GPS tracks / positions will also be delivered. As of now it will be a simple file upload widget at the front end and a gps data processing module at the backend.
5. Project Goals and Timeline
Project Goals:
- To provide a python web2py backend module for management of geospatial data. Several python gis libraries will be used to achieve this.
- To provide a standardized GIS frontend capable of displaying maps and other geospatial data (tracks, locations, affected areas, etc.)
- Advanced widgets for a rich Web2.0 user interface
- Facility to upload and process gps data
Timeline:
- Work already undertaken - Nil so far, as I decided to apply quite late. But as I am very familiar with the technologies involved I can get productive immediately after discussing the technology to be used with my mentor / dev team.
- First trimester (20 April - 22 May) - This period will be used mostly for evaluating and arriving at the technologies most suitable for the project. Evaluate backend gis tools, discuss, preferably write experimental code and arrive at a decision. Similar exercise will be carried out for the frontend where required. Python modules for gps processing will be explored. Evaluate OpenGTS and bring out issues of integrating it.
- Second trimester (23 May - 6 July)
- Implement experimental gis backend by Jun 8
- Code review, fixes and unittests by Jun 21
- GIS database integration (PostGIS, MysqlSpatial, SpatialLite) by Jul 5
- Third trimester (7 July - 10 August)
- Unittests and Integration tests for database backend and backend documentation by Jul 12
- Mockups for frontend map, controls and style up for review by Jul 19
- Basic map with layer switcher, pan and zoom control by Jul 26
- GPS data upload facility by Aug 2
- Frontend Extjs widgets, cleanup, finalization and documentation by Aug 10
- YUI Tests / Test Another Way based JS Tests by Aug 16
[1] http://code.google.com/soc/2008/turbogears/appinfo.html?csaid=E94B59193DB4A185
[2] http://www.turbogears.com/2.0/docs/main/Extensions/index.html#geo
[3] http://featureserver.org/
[4] http://pypi.python.org/pypi/OWSLib/0.3
Monday, August 25, 2008
TurboGears GSoC 2008 Final Report - Sanjiv Singh
Final status report for the project:
"ToscaWidgets Library for jQuery & OpenLayers"
GSoC 2008
Mentoring Org: TurboGears
Student: Sanjiv Singh
Mentor: Christopher Perkins
Dear Community,
The coding for GSoC officially ends today at 19:00 UTC. Before I move on to the report, I wish to convey my sincere thanks and gratitude to my mentor Christopher Perkins[1] who showed faith in me and constantly guided and encouraged me to move ahead. I would like to specially thank TG2[2] project lead Mark Ramm and ToscaWidgets[3] creator Alberto Valverde who have helped me a lot during the project. I would also like to thank our GSoC Admin Christopher Arndt, Michele Bertoldi, Laureano Arcanio, Bruno Melo and rest of the TurboGears Community for their help and support.
I also thank Eric Lemonie of Camp to Camp for helping me move ahead with MapFish and to the entire MapFish and OpenLayers Community for their support. I thank Christopher Schmidt of Metacarta Labs for OpenLayers and TileCache.
Although the initial project accepted for GSoC involved creating ToscaWidgets for OpenLayers and jQuery, after creating several widgets with jQuery and ExtJS, it was felt that some work on bringing in server side geo components to TG2 along with OpenLayers would enable the use of TG for GIS Applications and the TurboGears Extension tg.ext.geo project was born. At the end of coding for GSoC 2008 the status of all these works are as follows:-
1) Widgets for jQuery and ExtJS
Several widgets were created for jQuery and ExtJS. These include jQuery AjaxForm, jQuery Treeview, FlexiGrid, ExtJS SingleSelectCombo, ItemSelector (aka SelectShuttle). The documentaion and examples for these widgets with TG2 were in the old wiki documentation. They would be moved into the official TG2 ToscaWidgets Cookbook docs soon.
2) Widgets for JSUnit[4]
While coding for these widgets, a need for javascript unit testing tool was felt and a widget for JSUnit Javascript Unit Testing framework was developed. A runner widget for in-browser running of the tests was also created. The documentation for TG2 usage would be moved from old wiki to official docs soon.
3) OpenLayers[5]
A ToscaWidgets library has been created for OpenLayers Javascript Mapping toolkit. At present it has support for maps with layers of GoogleMaps, YahooMaps, VirtualEarth, WMS, OpenStreetMap layers using Mapnik and OSMRender, and Vector Layers (GML). The standard map control components like layer switcher, panzoom, etc. have been added. Controls for vector editing are also supported. A demo application with WMS and GML layers and vector editing of features can been seen at http://sanjiv.homelinux.net:8081/ . Feel free to edit the vectors and deform the geography as I have a backup of the db ;)
4) TG2 Geo Extension with MapFish[6]
The tg.ext.geo extension implements the server side component for vector query and editing. It is mainly based on MapFish server. However the templates are modified to suit the TG Object Distach BaseController. This may change in future and be fully based on MapFish server which uses a RESTFul controller. The extension makes use of existing PythonGIS tools like Shapely and GeoJSON. Thanks to Eric Lemonie, Sean Gillies, Matt Russell and the GISPython community for bringing the GIS tools to python developers. A demo application with openlayers based feature selection and a DojoGrid for displaying the attribute data can be seen at http://sanjiv.homelinux.net/ . Thanks to Michele Bertoldi for helping me with his Dojo Grid.
5) TG2 Geo Extension for TileCache[7]
tg.ext.geo has a paster command for reading a tilecache config file and generating controller code with TileCache mounted as a WSGI app in TG2. This makes it possible to use TileCache within TG for caching of WMS tiles using the WMS-C standard proposed by MetaCarta Labs. Once again thanks to Christopher Schmidt for creating TileCache. A demo app with backend tile generation using TileCache running in a TG2 app can be seen at http://sanjiv.homelinux.net:8082/ .
Over the next couple of weeks I would write tests and documentation for all these components and would continue to develop and maintain them in future as these works are far from complete. I would be extremely happy to receive your views, suggestions, feedbacks, etc. I would be most delighted to see people join in and move these tools forward.
Thanks once again to the open source community and Google for this great learning experience.
Regards
Sanjiv
[1] http://www.percious.com
[2] http://turbogears.org
[3] http://toscawidgets.org
[4] http://jsunit.net
[5] http://openlayers.org
[6] http://mapfish.org
[7] http://tilecache.org
"ToscaWidgets Library for jQuery & OpenLayers"
GSoC 2008
Mentoring Org: TurboGears
Student: Sanjiv Singh
Mentor: Christopher Perkins
Dear Community,
The coding for GSoC officially ends today at 19:00 UTC. Before I move on to the report, I wish to convey my sincere thanks and gratitude to my mentor Christopher Perkins[1] who showed faith in me and constantly guided and encouraged me to move ahead. I would like to specially thank TG2[2] project lead Mark Ramm and ToscaWidgets[3] creator Alberto Valverde who have helped me a lot during the project. I would also like to thank our GSoC Admin Christopher Arndt, Michele Bertoldi, Laureano Arcanio, Bruno Melo and rest of the TurboGears Community for their help and support.
I also thank Eric Lemonie of Camp to Camp for helping me move ahead with MapFish and to the entire MapFish and OpenLayers Community for their support. I thank Christopher Schmidt of Metacarta Labs for OpenLayers and TileCache.
Although the initial project accepted for GSoC involved creating ToscaWidgets for OpenLayers and jQuery, after creating several widgets with jQuery and ExtJS, it was felt that some work on bringing in server side geo components to TG2 along with OpenLayers would enable the use of TG for GIS Applications and the TurboGears Extension tg.ext.geo project was born. At the end of coding for GSoC 2008 the status of all these works are as follows:-
1) Widgets for jQuery and ExtJS
Several widgets were created for jQuery and ExtJS. These include jQuery AjaxForm, jQuery Treeview, FlexiGrid, ExtJS SingleSelectCombo, ItemSelector (aka SelectShuttle). The documentaion and examples for these widgets with TG2 were in the old wiki documentation. They would be moved into the official TG2 ToscaWidgets Cookbook docs soon.
2) Widgets for JSUnit[4]
While coding for these widgets, a need for javascript unit testing tool was felt and a widget for JSUnit Javascript Unit Testing framework was developed. A runner widget for in-browser running of the tests was also created. The documentation for TG2 usage would be moved from old wiki to official docs soon.
3) OpenLayers[5]
A ToscaWidgets library has been created for OpenLayers Javascript Mapping toolkit. At present it has support for maps with layers of GoogleMaps, YahooMaps, VirtualEarth, WMS, OpenStreetMap layers using Mapnik and OSMRender, and Vector Layers (GML). The standard map control components like layer switcher, panzoom, etc. have been added. Controls for vector editing are also supported. A demo application with WMS and GML layers and vector editing of features can been seen at http://sanjiv.homelinux.net:8081/ . Feel free to edit the vectors and deform the geography as I have a backup of the db ;)
4) TG2 Geo Extension with MapFish[6]
The tg.ext.geo extension implements the server side component for vector query and editing. It is mainly based on MapFish server. However the templates are modified to suit the TG Object Distach BaseController. This may change in future and be fully based on MapFish server which uses a RESTFul controller. The extension makes use of existing PythonGIS tools like Shapely and GeoJSON. Thanks to Eric Lemonie, Sean Gillies, Matt Russell and the GISPython community for bringing the GIS tools to python developers. A demo application with openlayers based feature selection and a DojoGrid for displaying the attribute data can be seen at http://sanjiv.homelinux.net/ . Thanks to Michele Bertoldi for helping me with his Dojo Grid.
5) TG2 Geo Extension for TileCache[7]
tg.ext.geo has a paster command for reading a tilecache config file and generating controller code with TileCache mounted as a WSGI app in TG2. This makes it possible to use TileCache within TG for caching of WMS tiles using the WMS-C standard proposed by MetaCarta Labs. Once again thanks to Christopher Schmidt for creating TileCache. A demo app with backend tile generation using TileCache running in a TG2 app can be seen at http://sanjiv.homelinux.net:8082/ .
Over the next couple of weeks I would write tests and documentation for all these components and would continue to develop and maintain them in future as these works are far from complete. I would be extremely happy to receive your views, suggestions, feedbacks, etc. I would be most delighted to see people join in and move these tools forward.
Thanks once again to the open source community and Google for this great learning experience.
Regards
Sanjiv
[1] http://www.percious.com
[2] http://turbogears.org
[3] http://toscawidgets.org
[4] http://jsunit.net
[5] http://openlayers.org
[6] http://mapfish.org
[7] http://tilecache.org
Subscribe to:
Comments (Atom)