The need of the hour for Google App Engine : Better adoption and more paying customers

September 15, 2011

As a developer who is betting on GAE (Google App Engine), the increase in hosting costs due to the recent price changes in GAE came as a surprise to me, as it was to many others too.

Developers who had optimized for the earlier pricing model which focused on CPU based pricing had to now deal with the new pricing model based on instance based pricing. Google realized (a little very late, in my opinion) that its old pricing model was flawed and instances which were alive but not consuming CPU (typically waiting on I/O) were costing more.

So Google had to change the model to match their actual costs, by moving to an instance-based pricing model, which by the way, is also how most other PAAS/IAAS service-providers charge.

As one of the Google developers details in the Google Groups..

..the price changes are a reflection of certain key facts:

a. Google as a company backing the entire platform as a product… instead of being cancelled, we’re coming out of preview mode and become an official product! Google is not a non-profit company and cannot continue to operate services at a loss. our products, and my paycheck’s gotta come from *some*where! coming out of preview means Google is committed to App Engine, and in turn, we’re committed to our users.

b. this service costs the company significant resources… premium services like App Engine and YouTube require a lot of hardware and networking bandwidth. We serve more than 1.5 *billion* pages views a day across all applications!

But Google did listen to the feedback from the GAE developers and addressed their concerns by increasing the free quota from 24 instance-hours/day to 28 instance-hours/day, while extending the 50% discount for instance hours up to Dec 01, 2011, when multi-threading support for Python is likely to be rolled in. This is because, thread-safe applications can serve more requests per instance, which currently is not possible for Python developers, while Java developers already have multi-threading support.

So, coming to the main issue, to remain a sustainable service, Google App Engine needs to have better adoption and more paying customers. Or in simple terms, Google App Engine needs to make money. For this, the developers betting on Google App Engine need to build apps which can solve business needs and be able to sell those apps easily.

There are 2 options available for developers to sell apps built on Google App Engine.

First option is to build the app as a multi-tenant application — segmenting data and restricting access to that segment of data based on the user who accesses the app, a little hard for those not used to multi-tenant apps, but doable — and to charge customers based on tiered-plans, the easiest approach being user-based plans. But this model of charging based on users is so very out-dated and in my personal opinion, a bit unfair too, because resource usage (and hence the costs incurred) need not directly be based on number of users accessing the app.

And the current pricing model changes in Google App Engine further complicates pricing for multi-tenant apps, as there is no way to allocate more computing instances to say, a premium customer or to restrict instances for customers in a basic plan. Even when apps start to meter requests, it is hard to allocate resources in a fair way,  especially when having to deal with instances and latency.

That brings us to the second option, which is for the apps to be installed directly into the customer’s Google App Engine account, for which the customer can purchase hosting resources directly from Google. This option is possible even now, but involves a lot of manual steps — which involves the customers to sign up for Google App Engine, create an application instance, invite the developer who then has to disable code downloads and then upload the code to that application instance — and hence is not very scale-able.

The second option needs to be automated so that any customer can visit a Marketplace for Google App Engine applications, read reviews about the apps and when they like an app, can just click on an “Install Now” button, to have the app automatically installed within their Google App Engine account. This marketplace can also be seamlessly integrated with Google Apps Marketplace, making it easy to install the app as a service directly into the Google App Engine account associated to Google Apps for their domain.

There can be a trial period for such an installed app, after which customers can make a payment ($9/month minimum + additional usage charges) so as to continue using the app, of which Google pays the developer a percentage.

This has now been filed as a feature request :
http://code.google.com/p/googleappengine/issues/detail?id=5821

Below is a quick list of past threads in the developer forums, which have been requesting for a Marketplace for Google App Engine apps, in one form or other..

Clarification of Term 4.4

How to distribute my app?

Questions about the preferred way to handle multiple apps/accounts

‘Add It Now’ Functionality for my own Google App Engine Application

Distributing the same app to multiple domains

Per-domain data stores and selling our apps to people who use google domains…

Is it possible to sell a CMS hosted on Google App Engine without the code being visible ?

Deployment API

Multiple Instances of the Same App

Selling App Engine Apps

Isolated Application Deployment Instances

Creating a Windows desktop deployment utility ie. port of appcfg.py to C# client library

Automating deployment

source code encryption

deploying applicaiton to multiple customers 

1 application, multiple datastores

usage & billing and multiple deployment
 
 
This is just a quick list. There should be more such requests. But it is obvious that this is an important feature which can enable better adoption rates of the Google App Engine, helping both the Google folks building this wonderful platform and the developers who love working on it.
 
If you have not already done so, you can star this feature request: Marketplace for Google App Engine apps, to let Google know that you are interested too.
 
While you can be sure that the custom online database application builder that I am working on will be available on such a marketplace, you can start using it today at http://creator.ifreetools.com.
Advertisements

“How to become an expert in my core skill (say Java) ?”

June 2, 2010

“How to become an expert in my core skill (say Java) ?”, I was asked recently by someone eager to become one, obviously.

Had sent the following reply, based on my personal experience..

With respect to programming, first find a project that you believe is interesting to work on.

Then think about..
How to write less code to add new features ?
How to reduce existing source code ?
How can things be made faster ?
How can things be made easier to work with ?

Each one is a challenge, which once achieved will make you want more such challenges. And, after some time, you will find people calling you an expert.. but who cares.. that is just a side benefit… you would have enjoyed the process anyway.

Good luck.

From the reply to my mail,…

How much knowing the API matters ? say JAVA has something called “Reflection” which i don’t know until now. If there is a problem which can be solved using Reflection i can use it right ? without knowing it i will be re-inventing the wheel. How people get to know that this is already available with API itself no need to write any custom code.

In java i actually read some books like ***, *** materials etc but still i didn’t know that this exists.

Valid concern.

Here was my reply..

Once you are working on a specific challenge, you will start looking for solutions, discussing with people who you think are experts or in online forums and checking out open-source tools which can help you. You will invariably end up learning what is required to get it done.

You may even end up discovering better approaches involving newer techniques / APIs, while also learning about how it was done until now.

Regarding reinventing the wheel. Remember the goal is to write as little code as possible. So, you would avoid doing it and if in a rare case you end implementing it with little coding.. great !! Now, compare it with the existing code – when you discover that it exists – and learn if your code is good or may be, better.

Feel free to add to the discussion with your comments.

After more than 8 years in Java at an enterprise software co., I am now enjoying interesting challenges with Python over Google App Engine, for my startup. You may check out my work at iFreeTools CRM and iFreeTools Creator – an Online Database Application Builder. You can reach me at raj@sahasvat.com.


PageSpeed is good for your website

May 3, 2010

I was procrastinating on an important feature in Chennai Bus Routes and Suburban Trains Information website, contemplating numerous options on how to implement it, since I feared it can make me go over the free quota in my Google App Engine deployment.

The feature was to enable users to identify hopping/switching points to travel between localities which are not directly connected by MTC buses / suburban trains. This planned feature was going to be a bit resource intensive – either with CPU, if done dynamically or with storage-and-CPU if it is to be indexed and synced periodically for all possible travel combination. I decided to take the CPU route and go the indexing route later, if required.

But, Google Webmaster informed me that even without the planned feature I was already very slow compared to a majority of the websites out on the internet.

So, using PageSpeed, I looked for the culprits.

The issues, apart from loading of a CSS file with lot of unused styles and the loading of separate JS file – both served from my Google App Engine instance, was the loading of numerous third party javascript files and their dependents. These included files from StatCounter Analytics, Google Analytics, Google Adsense (for 3 ads), Google Maps and AddThis sharing widget.

So, I set about fixing the issues..

  • I first removed unused CSS and JS, reduced their size and embedded them within the web page – 2 lesser requests to the server and reduced bandwidth.
  • Enabled memcache of full web-pages – this was useful, because for each page being loaded, Mediapartners-Google,gzip(gfe),gzip(gfe) (Adsense) was making a request for the same page again within a few milliseconds.
  • Next, I removed AddThis and replaced it my own sharing code and icon with just Twitter, Facebook and Delicious links. This would mean limited sharing options and no sharing stats, but I felt it was OK.
  • Removed StatCounter code from the main pages (routes for locality and route details pages) – now on, it will not be possible to easily see the sequence of pages visited, for recent visitors.
  • Reduced the number of Google AdSense ads from a maximum of 3 per page to just 1 – possibly lesser revenue from ads.
  • Increased the expiration time for static resources to 40 days – when those resources are to be changed, I will have to rename them to serve the latest files.
  • Delayed initializing and loading Google Map images, until the route details are requested – much better experience on slower connections and less cluttered look initially.

As a result, the page loading time came down a lot. Google Webmaster was quick to identify the changes in Page loading time (which was now approximately 30% of the original value) and the Kilobytes downloaded per day came down to 25% of original values, though the Pages crawled per day had actually increased a little during this time, as shown in the image below..
MetroCommute.in Crawl Stats

While I also added a new “find hopping/switching-point” feature, which did increase the usefulness of the website, I believe the page speed alone was responsible for better traffic (approximately 50% higher now) after these changes..

MetroCommute.in Google Analytics Dashboard

The reduced number of Adsense ads per page, down from 3 to just 1 per page, currently has a negative effect on the ad revenue, but not on the same magnitude. I believe it will grow better again with increased traffic. Apart from coming up higher for popular keywords, now more people are searching for variations of “metrocommute”. Which means they remember and recall the name of the useful website, which they had visited sometime earlier. Better brand recall, can mean letting go of Adsense and possibly direct CPM ads related to local travel services.

Some other options I did not look into, for improving the page speed are : Asynchronous tracking in Google Analytics, which I may take up later and reducing the actual page size (stripping out spaces, etc.,) – because the typical web page size is already very small, not even big enough to trigger gzip on Google App Engine.

Hope this post encourages you to look into making your website faster. Feel free to share your tips as comments.

Maintaining MetroCommute.in is more of a fun-project for me, while I work on iFreeTools, offering a free CRM app and an Online Database Application Builder over Google App Engine. You can reach me at raj@sahasvat.com.


Grails to iFreeTools CRM

October 21, 2009

Had to gain some domain knowledge on CRM/SFA (Sales Force Automation) and provide a solution for a client, to host in their premises.

My client was not so interested in Python and preferred the use of Java for the solution, more so after knowing my past experience. So after more than a year into Python was back into Java, evaluating frameworks for use in the project.

The search lead me to Grails.

The “Convention over Configuration” and “Don’t repeat yourself” approach which Grails enables was a pleasure to work with. (Except, where I believed that grails was against the DRY principles – having to define “constraints”, even for the sake of maintaining the order of properties.)

But, it was a lot better than having to code XML configuration files. One could easily configure/code Servlet Mappings, Filters and Tags, without having to add/modify a single XML entry. And even better, one could get a compiled war (with all those XML files) and deploy it in any servlet container !!

In the next week or so, building a CRM/SFA prototype using Grails helped me pickup stuff from both the domains. And, the way in which Grails works, made me want to approach Python programming for Google AppEngine with the “Convention over Configuration” and “Don’t repeat yourself” approach.

Though Django enables such features, I was a bit biased against using the full pack, since that would require packaging a “lot of files” – so much that it exceeded the file limits of initial Google AppEngine hosting and needed a new feature (zip packaging) to enable those features. I am allergic to “lot of files”, especially if I will have to modify a few of them to have it working my way.

And with the meta programming features available in python, I believed that it could be easier to implement a simple version for my requirements.

The result of that effort is : iFreeTools CRM (Alpha)


Diagrammr allows you to type in your diagram with sentences

August 27, 2009

Just add sentences and to get a diagram.

The following was generated for relations typical within an NMS (Network Management System)..

webnms

The sentences used to build were…

WebNMS manages ManagedObjects
NetworkElements are ManagedObjects
Interfaces are ManagedObjects
Switches are NetworkElements
Routers are NetworkElements
NetworkElements contain one or more Interfaces
Printers are NetworkElements
WebNMS communicates using Protocol
SNMP is a Protocol
WMI is a Protocol
JMX is a Protocol
NetworkElements may run Services
Services are ManagedObjects
NetworkElements support Protocol

Good tool, this diagrammr !!


Upgraded to Reliance Netconnect Broadband+

August 3, 2009

From my slow speed Reliance Netconnect, which was used as a backup for the Hathway broadband connections at home and office, I upgraded to Reliance Netconnect Broadband+ last week. This was after Hathway went down because of issues with politically advantaged competitor.

Considering that it I have replaced 3 internet connections with one (2 Hathway and 1 Reliance old-gen wireless modem), this works out to be cheaper (even when ignoring the discounted rates applicable for the first year) for my requirements. But, now I don’t have a backup service provider.

Short Review ::

PLUS : There were no installation issues with the current device (unlike my previous Reliance modem) and has a faster startup & good connection speeds.

MINUS : Reliance wants me to visit their website (rcom.co.in) every time I connect the modem – irritating behavior. And, unlike previous versions, it is not possible to make calls or send SMS using this modem.


How frequent are Chennai MTC Bus Routes ? MetroCommute.in provides the answers !!

July 8, 2009

Not even a month old, but we are happy to see MetroCommute providing answers to a lot of Chennai commuters. Thanks for your feedback and references.

In the mean time, we have also added support for showing the frequency of the bus services on individual routes, as well as the aggregate frequency.

Apart from finding interesting trivia – like the most frequent Chennai MTC bus route as 29C (Between Perambur R.S and Besant Nagar)  –

MetroCommute.in provides you information like..

  • there is one bus every 10 minutes from Velachery to Mylapore,
  • there is one bus every 4 minutes from Velachery to Ambattur and Adyar,
  • there is one bus every 2 minutes from Velachery to Tambaram, Thiruvanmiyur and T.Nagar and that
  • there is one bus every minute from Velachery to Saidapet and Guindy.

One can also know that…

  • there is one bus every 2 minutes on ECR,
  • there is one bus every minute on OMR and that
  • there is more than one bus every minute on Anna Salai.

These facts can be known by usual commuters on those routes. But if one has to travel on a different route, they may not be sure on the frequency. MetroCommute helps people decide on “Should I wait here for a bus or should I catch some bus to a more frequent stage and switch buses there ?”

Just play with http://my.metrocommute.in to find such answers related to your locality, refer/tweet/blog about the site to your friends and as always, feel free to send in your feedback on how to improve MetroCommute.in

And now with the onset of the counselling season in Chennai, MetroCommute helps plan a better commute…