The need of the hour for Google App Engine : Better adoption and more paying customers

September 15, 2011

As a developer who is betting on GAE (Google App Engine), the increase in hosting costs due to the recent price changes in GAE came as a surprise to me, as it was to many others too.

Developers who had optimized for the earlier pricing model which focused on CPU based pricing had to now deal with the new pricing model based on instance based pricing. Google realized (a little very late, in my opinion) that its old pricing model was flawed and instances which were alive but not consuming CPU (typically waiting on I/O) were costing more.

So Google had to change the model to match their actual costs, by moving to an instance-based pricing model, which by the way, is also how most other PAAS/IAAS service-providers charge.

As one of the Google developers details in the Google Groups..

..the price changes are a reflection of certain key facts:

a. Google as a company backing the entire platform as a product… instead of being cancelled, we’re coming out of preview mode and become an official product! Google is not a non-profit company and cannot continue to operate services at a loss. our products, and my paycheck’s gotta come from *some*where! coming out of preview means Google is committed to App Engine, and in turn, we’re committed to our users.

b. this service costs the company significant resources… premium services like App Engine and YouTube require a lot of hardware and networking bandwidth. We serve more than 1.5 *billion* pages views a day across all applications!

But Google did listen to the feedback from the GAE developers and addressed their concerns by increasing the free quota from 24 instance-hours/day to 28 instance-hours/day, while extending the 50% discount for instance hours up to Dec 01, 2011, when multi-threading support for Python is likely to be rolled in. This is because, thread-safe applications can serve more requests per instance, which currently is not possible for Python developers, while Java developers already have multi-threading support.

So, coming to the main issue, to remain a sustainable service, Google App Engine needs to have better adoption and more paying customers. Or in simple terms, Google App Engine needs to make money. For this, the developers betting on Google App Engine need to build apps which can solve business needs and be able to sell those apps easily.

There are 2 options available for developers to sell apps built on Google App Engine.

First option is to build the app as a multi-tenant application — segmenting data and restricting access to that segment of data based on the user who accesses the app, a little hard for those not used to multi-tenant apps, but doable — and to charge customers based on tiered-plans, the easiest approach being user-based plans. But this model of charging based on users is so very out-dated and in my personal opinion, a bit unfair too, because resource usage (and hence the costs incurred) need not directly be based on number of users accessing the app.

And the current pricing model changes in Google App Engine further complicates pricing for multi-tenant apps, as there is no way to allocate more computing instances to say, a premium customer or to restrict instances for customers in a basic plan. Even when apps start to meter requests, it is hard to allocate resources in a fair way,  especially when having to deal with instances and latency.

That brings us to the second option, which is for the apps to be installed directly into the customer’s Google App Engine account, for which the customer can purchase hosting resources directly from Google. This option is possible even now, but involves a lot of manual steps — which involves the customers to sign up for Google App Engine, create an application instance, invite the developer who then has to disable code downloads and then upload the code to that application instance — and hence is not very scale-able.

The second option needs to be automated so that any customer can visit a Marketplace for Google App Engine applications, read reviews about the apps and when they like an app, can just click on an “Install Now” button, to have the app automatically installed within their Google App Engine account. This marketplace can also be seamlessly integrated with Google Apps Marketplace, making it easy to install the app as a service directly into the Google App Engine account associated to Google Apps for their domain.

There can be a trial period for such an installed app, after which customers can make a payment ($9/month minimum + additional usage charges) so as to continue using the app, of which Google pays the developer a percentage.

This has now been filed as a feature request :
http://code.google.com/p/googleappengine/issues/detail?id=5821

Below is a quick list of past threads in the developer forums, which have been requesting for a Marketplace for Google App Engine apps, in one form or other..

Clarification of Term 4.4

How to distribute my app?

Questions about the preferred way to handle multiple apps/accounts

‘Add It Now’ Functionality for my own Google App Engine Application

Distributing the same app to multiple domains

Per-domain data stores and selling our apps to people who use google domains…

Is it possible to sell a CMS hosted on Google App Engine without the code being visible ?

Deployment API

Multiple Instances of the Same App

Selling App Engine Apps

Isolated Application Deployment Instances

Creating a Windows desktop deployment utility ie. port of appcfg.py to C# client library

Automating deployment

source code encryption

deploying applicaiton to multiple customers 

1 application, multiple datastores

usage & billing and multiple deployment
 
 
This is just a quick list. There should be more such requests. But it is obvious that this is an important feature which can enable better adoption rates of the Google App Engine, helping both the Google folks building this wonderful platform and the developers who love working on it.
 
If you have not already done so, you can star this feature request: Marketplace for Google App Engine apps, to let Google know that you are interested too.
 
While you can be sure that the custom online database application builder that I am working on will be available on such a marketplace, you can start using it today at http://creator.ifreetools.com.

“How to become an expert in my core skill (say Java) ?”

June 2, 2010

“How to become an expert in my core skill (say Java) ?”, I was asked recently by someone eager to become one, obviously.

Had sent the following reply, based on my personal experience..

With respect to programming, first find a project that you believe is interesting to work on.

Then think about..
How to write less code to add new features ?
How to reduce existing source code ?
How can things be made faster ?
How can things be made easier to work with ?

Each one is a challenge, which once achieved will make you want more such challenges. And, after some time, you will find people calling you an expert.. but who cares.. that is just a side benefit… you would have enjoyed the process anyway.

Good luck.

From the reply to my mail,…

How much knowing the API matters ? say JAVA has something called “Reflection” which i don’t know until now. If there is a problem which can be solved using Reflection i can use it right ? without knowing it i will be re-inventing the wheel. How people get to know that this is already available with API itself no need to write any custom code.

In java i actually read some books like ***, *** materials etc but still i didn’t know that this exists.

Valid concern.

Here was my reply..

Once you are working on a specific challenge, you will start looking for solutions, discussing with people who you think are experts or in online forums and checking out open-source tools which can help you. You will invariably end up learning what is required to get it done.

You may even end up discovering better approaches involving newer techniques / APIs, while also learning about how it was done until now.

Regarding reinventing the wheel. Remember the goal is to write as little code as possible. So, you would avoid doing it and if in a rare case you end implementing it with little coding.. great !! Now, compare it with the existing code – when you discover that it exists – and learn if your code is good or may be, better.

Feel free to add to the discussion with your comments.

After more than 8 years in Java at an enterprise software co., I am now enjoying interesting challenges with Python over Google App Engine, for my startup. You may check out my work at iFreeTools CRM and iFreeTools Creator – an Online Database Application Builder. You can reach me at raj@sahasvat.com.


PageSpeed is good for your website

May 3, 2010

I was procrastinating on an important feature in Chennai Bus Routes and Suburban Trains Information website, contemplating numerous options on how to implement it, since I feared it can make me go over the free quota in my Google App Engine deployment.

The feature was to enable users to identify hopping/switching points to travel between localities which are not directly connected by MTC buses / suburban trains. This planned feature was going to be a bit resource intensive – either with CPU, if done dynamically or with storage-and-CPU if it is to be indexed and synced periodically for all possible travel combination. I decided to take the CPU route and go the indexing route later, if required.

But, Google Webmaster informed me that even without the planned feature I was already very slow compared to a majority of the websites out on the internet.

So, using PageSpeed, I looked for the culprits.

The issues, apart from loading of a CSS file with lot of unused styles and the loading of separate JS file – both served from my Google App Engine instance, was the loading of numerous third party javascript files and their dependents. These included files from StatCounter Analytics, Google Analytics, Google Adsense (for 3 ads), Google Maps and AddThis sharing widget.

So, I set about fixing the issues..

  • I first removed unused CSS and JS, reduced their size and embedded them within the web page – 2 lesser requests to the server and reduced bandwidth.
  • Enabled memcache of full web-pages – this was useful, because for each page being loaded, Mediapartners-Google,gzip(gfe),gzip(gfe) (Adsense) was making a request for the same page again within a few milliseconds.
  • Next, I removed AddThis and replaced it my own sharing code and icon with just Twitter, Facebook and Delicious links. This would mean limited sharing options and no sharing stats, but I felt it was OK.
  • Removed StatCounter code from the main pages (routes for locality and route details pages) – now on, it will not be possible to easily see the sequence of pages visited, for recent visitors.
  • Reduced the number of Google AdSense ads from a maximum of 3 per page to just 1 – possibly lesser revenue from ads.
  • Increased the expiration time for static resources to 40 days – when those resources are to be changed, I will have to rename them to serve the latest files.
  • Delayed initializing and loading Google Map images, until the route details are requested – much better experience on slower connections and less cluttered look initially.

As a result, the page loading time came down a lot. Google Webmaster was quick to identify the changes in Page loading time (which was now approximately 30% of the original value) and the Kilobytes downloaded per day came down to 25% of original values, though the Pages crawled per day had actually increased a little during this time, as shown in the image below..
MetroCommute.in Crawl Stats

While I also added a new “find hopping/switching-point” feature, which did increase the usefulness of the website, I believe the page speed alone was responsible for better traffic (approximately 50% higher now) after these changes..

MetroCommute.in Google Analytics Dashboard

The reduced number of Adsense ads per page, down from 3 to just 1 per page, currently has a negative effect on the ad revenue, but not on the same magnitude. I believe it will grow better again with increased traffic. Apart from coming up higher for popular keywords, now more people are searching for variations of “metrocommute”. Which means they remember and recall the name of the useful website, which they had visited sometime earlier. Better brand recall, can mean letting go of Adsense and possibly direct CPM ads related to local travel services.

Some other options I did not look into, for improving the page speed are : Asynchronous tracking in Google Analytics, which I may take up later and reducing the actual page size (stripping out spaces, etc.,) – because the typical web page size is already very small, not even big enough to trigger gzip on Google App Engine.

Hope this post encourages you to look into making your website faster. Feel free to share your tips as comments.

Maintaining MetroCommute.in is more of a fun-project for me, while I work on iFreeTools, offering a free CRM app and an Online Database Application Builder over Google App Engine. You can reach me at raj@sahasvat.com.


Grails to iFreeTools CRM

October 21, 2009

Had to gain some domain knowledge on CRM/SFA (Sales Force Automation) and provide a solution for a client, to host in their premises.

My client was not so interested in Python and preferred the use of Java for the solution, more so after knowing my past experience. So after more than a year into Python was back into Java, evaluating frameworks for use in the project.

The search lead me to Grails.

The “Convention over Configuration” and “Don’t repeat yourself” approach which Grails enables was a pleasure to work with. (Except, where I believed that grails was against the DRY principles – having to define “constraints”, even for the sake of maintaining the order of properties.)

But, it was a lot better than having to code XML configuration files. One could easily configure/code Servlet Mappings, Filters and Tags, without having to add/modify a single XML entry. And even better, one could get a compiled war (with all those XML files) and deploy it in any servlet container !!

In the next week or so, building a CRM/SFA prototype using Grails helped me pickup stuff from both the domains. And, the way in which Grails works, made me want to approach Python programming for Google AppEngine with the “Convention over Configuration” and “Don’t repeat yourself” approach.

Though Django enables such features, I was a bit biased against using the full pack, since that would require packaging a “lot of files” – so much that it exceeded the file limits of initial Google AppEngine hosting and needed a new feature (zip packaging) to enable those features. I am allergic to “lot of files”, especially if I will have to modify a few of them to have it working my way.

And with the meta programming features available in python, I believed that it could be easier to implement a simple version for my requirements.

The result of that effort is : iFreeTools CRM (Alpha)


Diagrammr allows you to type in your diagram with sentences

August 27, 2009

Just add sentences and to get a diagram.

The following was generated for relations typical within an NMS (Network Management System)..

webnms

The sentences used to build were…

WebNMS manages ManagedObjects
NetworkElements are ManagedObjects
Interfaces are ManagedObjects
Switches are NetworkElements
Routers are NetworkElements
NetworkElements contain one or more Interfaces
Printers are NetworkElements
WebNMS communicates using Protocol
SNMP is a Protocol
WMI is a Protocol
JMX is a Protocol
NetworkElements may run Services
Services are ManagedObjects
NetworkElements support Protocol

Good tool, this diagrammr !!


Upgraded to Reliance Netconnect Broadband+

August 3, 2009

From my slow speed Reliance Netconnect, which was used as a backup for the Hathway broadband connections at home and office, I upgraded to Reliance Netconnect Broadband+ last week. This was after Hathway went down because of issues with politically advantaged competitor.

Considering that it I have replaced 3 internet connections with one (2 Hathway and 1 Reliance old-gen wireless modem), this works out to be cheaper (even when ignoring the discounted rates applicable for the first year) for my requirements. But, now I don’t have a backup service provider.

Short Review ::

PLUS : There were no installation issues with the current device (unlike my previous Reliance modem) and has a faster startup & good connection speeds.

MINUS : Reliance wants me to visit their website (rcom.co.in) every time I connect the modem – irritating behavior. And, unlike previous versions, it is not possible to make calls or send SMS using this modem.


How frequent are Chennai MTC Bus Routes ? MetroCommute.in provides the answers !!

July 8, 2009

Not even a month old, but we are happy to see MetroCommute providing answers to a lot of Chennai commuters. Thanks for your feedback and references.

In the mean time, we have also added support for showing the frequency of the bus services on individual routes, as well as the aggregate frequency.

Apart from finding interesting trivia – like the most frequent Chennai MTC bus route as 29C (Between Perambur R.S and Besant Nagar)  –

MetroCommute.in provides you information like..

  • there is one bus every 10 minutes from Velachery to Mylapore,
  • there is one bus every 4 minutes from Velachery to Ambattur and Adyar,
  • there is one bus every 2 minutes from Velachery to Tambaram, Thiruvanmiyur and T.Nagar and that
  • there is one bus every minute from Velachery to Saidapet and Guindy.

One can also know that…

  • there is one bus every 2 minutes on ECR,
  • there is one bus every minute on OMR and that
  • there is more than one bus every minute on Anna Salai.

These facts can be known by usual commuters on those routes. But if one has to travel on a different route, they may not be sure on the frequency. MetroCommute helps people decide on “Should I wait here for a bus or should I catch some bus to a more frequent stage and switch buses there ?”

Just play with http://my.metrocommute.in to find such answers related to your locality, refer/tweet/blog about the site to your friends and as always, feel free to send in your feedback on how to improve MetroCommute.in

And now with the onset of the counselling season in Chennai, MetroCommute helps plan a better commute…


Metro Commute in Chennai using buses or trains connecting West Mambalam

June 9, 2009

Anugraha Hall details in MatrimonyPlanner has the following “Access details” property..

ROAD :

From Rajiv Gandhi Road / OMR : Take road towards Velachery, just after Naidu Hall and Sony showroom, take left to reach Anugraha Hall.

From Vijaynagar Bus Terminus in Velachery, take Taramani Road and just after Baby Nagar Bus-stop, take right for Anugraha Hall.

TRAIN :

Nearest MRTS Station : Perungudi. You may take an auto-rickshaw or walk (refer our map for details). 

And this does not even include details on how to get to Velachery (from say Tambaram / Central / Egmore / Koyambedu / etc.,.). You would have noticed that wedding cards usually have the bus routes to reach to the mandapam locality and it imperative to have better details as part of the marriage hall details in MatrimonyPlanner.

The following is Sahasvat’s attempt towards that goal..

Buses-or-Trains/Connecting/Velachery

Buses-or-Trains/Connecting/Velachery/with/Central

Buses-or-Trains/Connecting/Velachery/with/Koyambedu

Buses-or-Trains/Connecting/Velachery/with/Tambaram


MetroCommute.in

 

The details provided in the website include bus routes and train routes to localities in and around Chennai.

The website is still in beta and may need some more ironing out of details with respect to stage names of bus routes and their geo-locations. This is because the data from the MTC bus site, cannot be used as is. There can be a lot of ambiguity and some incomplete/incorrect stage names..

To give you a sample on ambiguity..

  • “Madipakkam Rd. Jn” may mean the road junction in Madipakkam or the one near Velachery MRTS.
  • “Anna Nagar” may also refer to Anna Nagar near Korukkupet or to the Anna Nagar near Vandalur.
  • “Ramapuram” may also refer to Ramapuram near Putlur.
  • “Thirumangalam” may also refer to Thirumangalam Village, near Sunguvarchathiram.
  • “Brindavan Colony” may be Brindavan Colony in Adambakkam or the one near Anna Nagar.
  • “Thandalam Village” near Thirupporur is actually Mettu Thandalam Village.

And a sample on incomplete/incorrect stage names..

  • “Andakuppam” may actually refer to Andalkuppam, near Manali and different from Andakuppam near Kundrathur. 
  • There are stage names like P135, E022, C064, K115, V085, G023, K116, etc.,.

For most errors, I would assume the reason was some sort of an auto-completion/selection being used when entering the stage-names. But I wonder why stage codes (like P135, E022, etc.,) have to be left in the routes. I guess, the first letter in codes should actually refer to the starting letter of the stage name.

The ambiguous nature of the stage names can cause route maps to have stages out of path, but easy to identify visually, when loading the route map. Once that is done, the alternate/name of the stage and its exact location can be found by zooming into the region and doing a search. This can still fail to identify the location (probably due to differences in spelling – “c” instead of “s”, an extra “e” or “h” or “l” or “t”,  “d” instead of “t” or i/y/e used interchangeably or differences in using spaces, etc.,.). In such cases one needs to switch to the hybrid view and see for settlements in that locality and then find their names.  Even this can fail as many will have just “the great abc’s home” / “xyz’s plot” kind of descriptions. For some remote locations I may have interpolated a few stage locations. A few more iterations and possibly feedback from users may be required to have even better accuracy.

Suburban train route details from Southern Railway did not have such issues.

Driving directions and call-taxi / call-auto / call-driver details will be coming up next.

Details on switching buses en-route, may not be taken up at once. It is not a trivial problem to solve and while it is easy to implement it in a simpler way, it can fail providing wrong suggestions. For example, if you search for options from Velachery to Chromepet, you will be asked to travel to Thiruvottiyur Bus Stand, in one of the existing solutions. 

“Wait, you mean there are already websites which use the MTC bus routes data.. then, why do you have to build MetroCommute.in ? How different is it from others ?” you might ask.

MetroCommute.in site is not just for MTC bus routes. Details of existing suburban train routes including MRTS, Beach-Tambaram, Tambaram-Chengelpattu, Central-Thiruvallur, Central-Arakkonam, Central-Gummidipoondi and Central-Sullurpeta lines are already part of the site. Details of Call-Taxi / Call-Auto and Call-Driver numbers, Personal driving directions, Discussion Threads on commuting options, Group Travel, etc., will also be supported. The goal is when you come to the website, your commuting option should be finalized.

(By “Group Travel” I refer to something like an ondemand carpool. This is already common in offices, where employees form a group and book a Sumo/Van for attending a colleague’s wedding and share the costs. But what if the wedding is for someone who is not a colleague, but people from your locality are attending the same. And, what if more people are travelling to the same locality, but for a different wedding. Well that is just a rough idea.)

MetroCommute.in takes a locality-centric approach to finding the routes, rather than limiting itself to specific bus stages. This is more useful when you are not sure of the stage names (like for example, there is no stage name called “Ashok Nagar” or “West Mambalam” in the MTC bus stages), but with MetroCommute.in you can find..

Buses-or-Trains/Connecting/Ashok Nagar

Buses-or-Trains/Connecting/West Mambalam

Once you reach the locality, you can always use an auto-rickshaw for a shorter commute to your specific destination.

You can even search for options like..

Buses-or-Trains/Connecting/Mount Road/with/OMR

Buses-or-Trains/Connecting/ECR/with/OMR

MetroCommute.in is developed to be embedded into other sites which have locality specific data. Sahasvat has 3 such sites (MatrimonyPlanner.in, PlaceToStay.in and PlaceToLive.in), where it requires to use this feature. The type of integration can also be flexible, with a possible two-way integration with MetroCommute.in. So building a site with such flexible options can be useful when it comes to integrating with other websites.

We welcome your feedback/suggestions on improving the usefulness of MetroCommute.in .


Buyer activism at DLF office – 300 converge seeking refund.

March 29, 2009

Around 300 buyers seeking to exit the DLF Garden City project converged at the DLF office on Saturday evening (28 March 2009) and refused to leave until DLF agreed to refund their money.

Details here :
Around 300 buyers converge at DLF office seeking refund – DLF agrees !!

300


Buyers : “DLF Garden City Rate Cuts Not Enough, Considering Exit Offer”

February 25, 2009

ET now reports that despite the rate cut announced by DLF for its Garden City project in Semmanchery (near OMR, Chennai), the buyers are considering the exit offer from DLF.

dlf-omr-commentsOne of the buyer seems to have commented on the TOI article “Builder slashes apartment prices on OMR” (which reported the price cuts), says that “GC google group members core demands have not been met and also this reduction is not even close to bangalore and hyderabad DLF projects. Everybody had expected atleast 20% from the soft launch price. Disappointed and will exit from the project.” (read comment on ToI)

According to buyers, the 13-month inordinate delay and launch of the project without proper sanctions had driven many away. DLF’s recent mail to the buyers claims to have got the DTCP approvals and that it will start the process of signing Apartment Buyers’ agreements very soon. It also claims that it can complete the construction by April 2011, as was initially planned.

The article is available at : 
http://economictimes.indiatimes.com/News-by-Industry/DLFs-exit..

Earlier Blog Post : 
DLF reduces apartment prices in Garden City, OMR !! 

Also read :
Discussion Thread on DLF Garden City (including earlier news articles).