Saturday, August 16, 2014

Hacking Kubernetes on Azure


Kubernetes got some press recently because Microsoft, IBM, RedHat and Google agreed to work together on a cloud platform.

I was looking into a similar platform at the time (Mesos/Marathon stack from Apache) which was good, but didn't quite pass my open source litmus test. (ie. Ability to hack a working build in under 5 minutes). I had a mesos cluster running, but I had to build the zookeeper binaries from source to get the multi-machine cluster working.

Kubernetes cluster was up and running locally without any trouble in vagrant environment.

git clone https://github.com/GoogleCloudPlatform/kubernetes
cd kubernetes
vagrant up

The one machine cluster was also up and running quite easily, but the nginx docker images didn't deploy because of port conflicts.
Then, I launched a cluster on GCE which also worked without any issues. So far, so good.

Why Azure? - I had MSDN credits that I could use.
I tried to use the azure scripts that were included, but it seemed to reference old files & didn't quite work.
It also turned out that Kubernetes was only tested on either Redhat/CentOS or debian.

Support for Ubuntu 14.04:
The installation of salt used bootstrap scripts that were not supported on 14.04 as per the website.
Instead, use the method recommended by saltstack:

apt-get install -y software-properties-common
add-apt-repository -y ppa:saltstack/salt
apt-get update
apt-get install -y salt-master salt-minion

The installation scripts for docker wasn't quite doing the job, I took the easy route on this one by disabling docker in cluster/saltbase/salt/top.sls.
Instead use the insecure one liner installation:

curl -sSL https://get.docker.io/ubuntu/ | sudo sh

Support for Azure:
Kubernetes seemed to make some assumptions about the private network, so I decided to use the configuration from vagrant.
Use the Azure console to create a network with the following configuration. (ensure the ip range and subnets are present)
AddressPrefix: 10.0.0.0/8
Subnet-1: 10.245.1.0/24
Subnet-2: 10.245.2.0/24

Run this to create a build and upload to Azure storage:
./release/build-release.sh 

Run this to bring up the cluster:
./cluster/azure/kube-up.sh

Once you have the cluster running, you can ssh to the master or minions:
ssh -i ~/.ssh/azure yourcloudserviceshere.cloudapp.net -p 22000

For reference, I posted a copy of the code that I now have running on Azure @:
https://github.com/vishnuvaradaraj/kubernetes

Overall, the platform looks promising - It was easy to define pods to deploy all kinds of docker containers including nginx load balancers for node.js, mysql, grafana etc; the etcd service from coreos seems better than zookeeper due to better API access (REST) and can easily scale to thousands of machines. The salt framework was interesting, easy to use once you figure out how to debug the basic issues.

Sunday, February 05, 2012

Google Appengine - Seriously Google, Don’t be evil!

It was good while it lasted, Google AppEngine gave developers an easy way to create web applications without worrying about scaling it or being a sysadmin. There were numerous startups that built their products on Google Appengine. Even after beta, there was a fairly large free quota that made it a reasonable choice. But, the current pricing seems a bit ridiculous for those who are looking for a free or cheap solution or for those enjoyed the initial quotas. Perhaps, we were spoiled and Google’s goal is to get rid of free-riders which is not exactly evil from Google’s perspective.

So, it is 2012 and welcome to reality since Google can’t maintain those quotas forever and the current free quotas are intended to only test your application before you go live. If for some strange reason, your application becomes popular and gets more than 1,000 users, you would end up scrambling for an alternate solution since you will find yourself paying through the nose. The quota system is complicated than is necessary from user's perspective and with constant changes to the billing policies it gets more confusing. This is even more painful since you probably have already worked around those SQL limitations in AppEngine store, quota restrictions and lack of file access, time restrictions etc. Frankly, save yourselves the trouble and gain control of your business. Stay away from Google’s AppEngine to begin with.

It is definitely useful for simple apps for your friends and small businesses with sufficient funding as long as you are not hugely successful. There are also open source alternatives to host Google Appengine code like AppScale, but then why bother? Why don’t you just start with a clean slate and pick the right technology stack that gives you the flexibility and control from day one! Obviously, don’t take this as the right answer for everyone, there is a market for the product which is probably why Google is able to raise the prices multiple times and still be successful. I had also seen similar blogs where people commented RTFM, why did you choose to stick with a locked in solution, were you stupid etc if you complained about GAE.

I am hoping this blog helps some small startups out there who is evaluating whether to use Google AppEngine stack or not for a complex web application. The startup companies usually have strong technical talent, but has limited cash. It is easy to get to 1,000 users, especially with free apps. Startups also aim to make profits by gaining millions of users, so beware of Google AppEngine. Don't lock your future to Google. Still, if you have limited users or you are willing to pay for AppEngine usage which is higher than other options available, then Google Appengine is the right choice because it offers a lot of convenience for the non-Linux savvy user. It is also good for quick prototyping as long as you are happy with the sandboxed environment.

Other options include Amazon EC2, AppScale, Windows Azure, Salesforce.com, Apache Hadoop, Joyent etc.

Finally, Google does seems to be having an evil streak nowadays with the change in privacy policy, Kenyan scandal, OpenMaps accusation, not indexing competitor content from Twitter, Facebook etc. It may be a sign of growing up, perhaps new start-ups should stay away from such mottos! , Sure, Google has done a lot of good things to developers like open sourcing everything that helps their business model (search code is still closed), Android is open sourced with a hidden whip, users got a lot of free web apps that in turn helped its search empire etc. I do have a lot of respect for Google as a technical company that gave us Pagerank, MapReduce, GFS, Maps etc. But, with AppEngine Google is squandering away the goodwill from a lot of successful developers and early adopters with regular "planned" price increases, system downtimes etc.

Monday, June 20, 2011

What is up with RIM?

RIM is one of the leading technology companies in Canada and one of the largest employers in Waterloo. There are a few strategic mistakes that seem obvious in hindsight, but it is easier to list them now than to predict in advance.

RIM is losing market share in North America, but surprisingly the execs were focusing on developing markets for growth. Obviously, profit margins from developing countries are minimal when compared to the lucrative NA market.

RIM’s strength is in the enterprise market. The primary growth is happening in consumer market, but RIM has had trouble gaining traction there. RIM will continue to rule the enterprise for now, because changes come at a slow pace there. Still, the consumerization of IT is the biggest threat for RIM in this area. The current instability doesn't help them advance in enterprise either.

RIM is losing the high end smart phone market which is ruled by Apple and Android devices, but instead is happy with the wins in mid-low end of the market. Again, this is likely to be short lived because cheap Android phones are just around the corner.

RIM’s strength is the security and convenience offered to business customers and the Blackberry brand represented by successful executives and presidents. But, RIM did nothing to promote these advantages, rather diluted the brand by issuing a lot of cheap phones for a short-term gain.

RIM chose to act on Apple envy rather than trying to innovate and differentiate itself, which was all the more important in the crowded and fiercely competitive smartphone market. This meant acquiring TAT, introducing Playbook etc, but also that RIM would trail Apple by at least one release.

RIM focused completely on the Playbook perhaps ignoring the Blackberry product line. The co-ceo at one point indicated that it is his ‘baby’ which is both good and bad. It is good because it shows passion, but could also be the root cause behind why RIM announced that all other product releases are delayed now.

RIM introduced the Playbook which was indeed the type of innovative initiatives that the company needed, but they did make a few strange decisions like using Flash for the UI. Many enterprises wouldn't even think of having Flash given its reputation on security. So, it was indeed a strange marriage of the 'most secure' OS with the 'most unsecure' UI technology. Perhaps, they should indeed have acquired Palm.

RIM expressed high confidence on the yearly guidance despite multiple indicators of slowed growth. RIM also had to revise the quarterly numbers multiple times. It clearly showed operational challenges at the top and it is easy to lose your credibility when such things happen.

RIM is highly optimistic about the 2nd half of the year for some mysterious reasons. It seems unlikely from the outside that a revamped Blackberry running the QNX platform will even be able to work such magic. But, then they are not even expected until early-2012 as per the earnings call.

RIM decided to pre-announce that all Blackberries would shift to QNX platform, it was better to keep it a rumour thus not affecting current Blackberry sales. Everyone knows that companies in transition are vulnerable. Execs are blaming the 'temporary' problem on Argentina etc, but then why do you layoff people if you don't foresee an extended downturn?

RIM acquired QNX in 2010 and was able to produce a new device running the OS and a new set of applications in less than an year. It is an unbelievable engineering feat and they had to pay a big price for that strategic decision in the form of bad reviews, poor quality and lack of features. eg. home page icons that open up the browser.

RIM’s strength is their employees and that is the only hope for a turnaround. The employees are largely graduates from the reputed UW, but who have probably not worked anywhere else. There is definitely a precedent where companies with talented workforce and good management have turned around, even though comparisons to Nortel and Nokia are floating around as well.

RIM unfortunately was caught up in the game of numbers played by Wall street analysts. The layoffs are just the standard operating procedure for meeting targets when things go wrong. But, they could have seen the weakness in market ahead, cut down on hiring, removed perks like free BB, salary hikes, cut 401k etc as well. But, the execs decided to play it by Wall street rules to make the analysts 'happy' - at this late stage?

RIM co-CEOs are indeed heroes who have built this $20 billion company from scratch. It is rather a strange management structure with two CEOs who are chairmen of the board as well. Sadly, their recent interviews and actions have also affected the company adversely. BBC and Mossberg interviews for example. Perhaps, it is a good option for them to take a break once this storm is blown over.

“strong, consistent leadership is critical to RIM at this juncture” – is partly correct, what RIM really needs at this juncture is fresh management that would instill confidence in the investors and bring revolutionary changes instead of “hoping” for the next batch of devices to solve all the problems. It should focus on its core strengths and leverage them for the new market requirements.

Tuesday, September 28, 2010

Blackberry Playbook

RIM introduced the "Playbook" tablet computer with much fanfare yesterday.It relies on Flash & HTML5 as the key development platforms. RIM was able to build the compatibility for existing apps on top of QNX which is good. It was smart to rely on the HTML5 platform given the poor UI capabilities in the existing Java framework, it wasn't clear if the tablet supported native game development despite the support for OpenGL in QNX platform.

The specs are great compared to the existing iPad, RIM did need some +ve hype at this point after the Torch fiasco. It is just 10mm thick compared to 13.4mm for the iPad and I thought iPad was thin. It has a 7" screen, but the resolution is 1024x600 whereas it is 1024x768 for the 9.7" iPad. Playbook also comes with a dual core 1GHZ processor compared to the single core CPU in iPad. Playbook also comes with front and back cameras and HD display with HDMI output compared with none for iPad. It also tethers with the Blackberry for 3G connection as opposed to the additional fees required for iPad. In short, RIM has upped the ante on the tablet front and we will probably see more from other Android OEMs in the short run.

The Playbook also has a few things going against it like the lack of native apps, GPS, 3G support and the introduction of a new OS which is not proven yet. The reliance on Blackberry for 3G can be both good and bad. Ultimately, the success would depend on how the marketing and messaging around this new device motivates app developers and users. eg. Palm OS was technicaly good on paper, still failed to succeed. The stock market reaction was not very strong either.

Related announcements on the software front also waived the fees for the developer program, introduced an ad platform (makes sense given that iPhone ad program has captures 1/5th of the mobile ad market) and open sourced the webworks platform. RIM is beginning to show leadership, but the key aspect is how it can integrate QNX into its legacy hardware and how it captures the consumer market. The playbook even though is technically advanced still plays to the enterprise market rather than the consumer market which has always been tough to crack for the Blackberry maker.

See this post for more details on specs etc.

--
Update 10/26.
RIM announced the SDK for Playbook development which is based on AIR. I was able to develop an app in under 30 minutes and deploy to the VM running QNX. There were no apps included in the OS, but there were some promising animations in the UI. You can also develop using Flex via the new Flash Builder Burrito beta which is pretty cool (even though it requires some workarounds). I'd rather wait for the upcoming HTML5 development option.

Tuesday, September 07, 2010

Mobile tech landscape

Just some personal thoughts from development and business perspectives:

Apple - iPhones and iOS; top of the line industrial design and beautiful user interfaces. Combine this with the maturity of the Mac OS platform, excellent native development tools and users willing to buy apps on AppStore this is a top platform to target. The requirement for Macs & to learn Objective-C are initial deterrents, but probably it just helps keep Windows developers away. The simple fact is that develepors will be where the money is and Apple Appstore is one place where open source hasn't yet taken away the revenues for indie developers. iPads have extended the success of iPhones, but has made it challenging for developers to target multiple screen sizes.

Android - This is one of the hottest mobile platforms; mainly because people equate this to PC vs Mac from the good old days. (open vs closed). The Android platform is indeed one of the most developer friendly mobile platforms out there, it is still Java based even though the native APIs are also open sourced. The level of freedom provided by the platform and the availability of source code are major advantages. The disadvantage for this platform is the variety of devices & screen sizes that needs to be supported and the fragmentation caused by some device vendors. Android 3.0 would also have better support for iPad like tablets.

Blackberry - This is the current platform of choice for business users, but it is not very attractive for consumers when compared to existing iOS & Android devices. Blackberry is a great "phone" and an "email reader". But, they are not the only features required to succeed in the smartphone race in consumer market. It recently upgraded the browser to Webkit which was long overdue. It's development platform is based on Java ME and produces chunky UI even if you try really hard not to. Blackberry OS 6.0 has improved, but still lacks several key features for the consumer market. eg. poor platform for games. In my personal opinion, RIM should have bought Palm, integrated WebOS UI and upgraded its CPU & hardware.

Palm - This was one of the over-hyped platforms that was technically superior, but failed to succeed as a business. HP bought Palm and is now coming up with the second version of WebOS. The platform is very developer friendly and is based on javascript. The developer workflow is similar to Ruby On Rails and is extremely productive. There is no clear differentitor for the product even though it is developer friendly which is probably why it failed to take off.

Microsoft - Windows Phone 7 is the latest entrant and is the underdog. Microsoft plans to spend a billion dollars promoting the new OS. Microsoft has recently lost the advantages it had in the business market via Windows Mobile as well. Unfortunately, Microsoft still doesn't seem to get it. For example, it is the only smartphone in the above list which doesn't include the Webkit browser. Perhaps, a standards compliant IE9 is good enough, but obviously it wouldn't include any of the webkit specific features that advanced mobile sites will make use of. But, the strategy is definitely in line with the Microsoft agenda of coming late to a market with proprietary stuff and taking over it. The native development platform is based on .NET/Silverlight technologies. In this case, no one expects Microsoft to succeed though.

Summary - Mobile devices & other touch devices are here to stay. Hardware is fast becoming a commodity nowadays. eg. what more can you pack in a smartphone except more power? :) So, there has to be some other differentiator for the winner. eg. apps, games, books, music, subscriptions. Smartphone platforms tend to be more sticky due to the above. Users might avoid switching platforms since they don't want to lose the apps or other content they have already bought rather than the hardware itself.

Unlike the open source free software mentality in the desktop world, there are a variety of appstores that help developers make money. But, the question is whether the money in the appstores are sufficient to sustain a business or not.
Apple customers are more willing to spend money on software than others, so its appstore would continue to attract developers for sometime even if the number of users are more for other platforms.
Still, it is widely expected that Android will succeed over the long run due to the sheer number of OEMs that produce devices based on Android.
Blackberry has to significantly change its strategy in the consumer market (instead of producing copycat devices) to succeed.
Microsoft will most likely be relegated to niche markets despite the marketing push.
Are developers going to produce native applications on all these platforms? Definitely not.

Monday, November 16, 2009

MBET - what do you learn in an MBA?

1) People don't often invest in ideas, people invest in people. eg. Heck, we are going to have fun together building 'x'.
2) Don't just create cool technology, instead produce something that solve a customer pain. hint: If they don't want to pay for it, then it is probably just a cool toy.
3) What you decide to not do is sometimes more important that what you decide to do.
4) Focus on making the pie bigger, not on cutting smaller pieces. ie. share equity if you end up with a bigger net value.
5) Hire smart, trustworthy people that share your passion, never ever hire a single poor performer or a bad team player. eg. A grader hires other A graders, but B grader hires other C graders.
6) Focus is better than hope. eg. targetting 1% of 50 million market is not a good goal.
7) Avoid raising money from friends & family; hint: few startups actually succeed.
8) Write a business plan, more to help you than to help others. Don't bother to create one if you don't understand what it is.
9) Don't start a business with another engineer if you can, business or a design person would probably add more value.
10) Money tends to amplifies your base characteristics. eg. if you are a good person, then it is good for the rest of us. But focus on value, not making money.
11) The networking & exposure is much more valuable than the classroom learning.
12) Learn how to interpret financial statements even if you don't plan to be a CA.
13) Strategy is a web of interconnected decisions made by a company that re-inforces each other.
14) A single idea can be copied, but it is much more difficult to copy a strategy due to the above reason.
15) Use valuation models used by VCs yourself, then you can reverse engineer how they value your company.
16) Lack of resources can be a competitive advantage, if you plan wisely.
17) And last but not the least, Believe in yourself!

Tuesday, October 28, 2008

Windows Azure - Microsoft's cloud platform

Microsoft doesn't seem to have changed it's strategy a little bit even though many have hailed this as a 180 degree turnaround. It is again copying what other startups and market creators have done (cloud computing, internet as a platform etc) and then executing on the plan so well that it is cheap(?), superior and accessible to everyone. At the same time, it also tries to win back the developers and defend its platform. So, what is new?

For those who missed the announcement, Microsoft came out with a comprehensive cloud computing platform for developers and organizations which can be used to host applications developed for the platform. To its credit, the platform is so comprehensive and well integrated that Google's equivalent AppEngine seems like.. well, something that was built on 20% time. (But, note that the Sql Data services seems awfully similar to the Google Bigtable model.) It also seems to have pulled the entire company including Office, SQLServer, Developer division & Windows platform around the idea which is indeed rare.

Some of the technical highlights are the use of a new hypervisor to host the client applications in VMs, evolution of SQLServer to SDS which is schema-less and provide a REST/SOAP/LINQ api, integration with Live & mesh services with a robust synchronization framework. Full SQL support is not expected due to it's inability to scale in distributed environment, but native apps will be supported shortly since hosting is based on VM technology.

The biggest competitor to Microsoft seems to be itself, since it is wooing ISVs but how would you convince someone to build on the platform when it seems to cover almost everything? And perhaps users who are wary of seeing BSOD in their internet boxes, but an impressive shift to cloud computing from a tech perspective.