tag:blogger.com,1999:blog-59881802024-03-14T01:49:32.746-07:00Vishnu Varadaraj's BlogRandom thoughts and notes...Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.comBlogger57125tag:blogger.com,1999:blog-5988180.post-76970227375408431192019-02-06T19:31:00.000-08:002019-02-07T20:10:27.342-08:00Did you ever have to analyze large log files before?
Well, I recently had to analyze a large log based dataset and decided to try out the ELK stack.
Introduction:
Please see this intro if you are not familiar with ELK, the rest of the blog assumes you know what they are. eg. elastic server, logstash & kibana. It is a popular server side tool to index, search & graph a large collection of logs or similar structured/unstructured data. Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com3tag:blogger.com,1999:blog-5988180.post-7329230474335315982016-02-21T19:18:00.002-08:002016-02-25T19:08:38.286-08:00Mobile development - to be native or not!
Whenever you start a new cross-platform project, there is always the question about whether to use native languages and tools for each platform or just build an HTML/javascript solution that works fine on all platforms.
The decision to use HTML/JS is easy for most teams - you have a single code base, you are working against a tight budget, you have hardly enough time to develop for one Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-88737002878729944722014-08-16T10:40:00.003-07:002014-08-16T16:57:05.618-07:00Hacking Kubernetes on Azure
Kubernetes got some press recently because Microsoft, IBM, RedHat and Google agreed to work together on a cloud platform.
I was looking into a similar platform at the time (Mesos/Marathon stack from Apache) which was good, but didn't quite pass my open source litmus test. (ie. Ability to hack a working build in under 5 minutes). I had a mesos cluster running, but I had to build the zookeeper Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com3Waterloo, ON, Canada43.4642578 -80.520409643.3720868 -80.681771099999992 43.5564288 -80.3590481tag:blogger.com,1999:blog-5988180.post-74789952878641117882012-02-05T10:45:00.000-08:002019-02-06T21:14:58.726-08:00Google Appengine
It was good while it lasted, Google AppEngine gave developers an easy way to create web applications without worrying about scaling it or being a sysadmin. There were numerous startups that built their products on Google Appengine. Even after beta, there was a fairly large free quota that made it a reasonable choice. But, the current pricing seems a bit ridiculous for those who are looking forVishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com5tag:blogger.com,1999:blog-5988180.post-48353514274471016472011-06-20T19:38:00.000-07:002019-02-06T21:16:34.001-08:00What is up with RIM?
RIM is one of the leading technology companies in Canada and one of the largest employers in Waterloo. There are a few strategic mistakes that seem obvious in hindsight, but it is easier to list them now than to predict in advance.
RIM is losing market share in North America, but surprisingly the execs were focusing on developing markets for growth. Obviously, profit margins from developing Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com2tag:blogger.com,1999:blog-5988180.post-16356580921673764772010-09-28T05:24:00.000-07:002019-02-06T21:16:44.185-08:00Blackberry Playbook
RIM introduced the "Playbook" tablet computer with much fanfare yesterday.It relies on Flash & HTML5 as the key development platforms. RIM was able to build the compatibility for existing apps on top of QNX which is good. It was smart to rely on the HTML5 platform given the poor UI capabilities in the existing Java framework, it wasn't clear if the tablet supported native game development Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-11414964236680162482010-09-07T08:18:00.000-07:002019-02-06T21:17:07.920-08:00Mobile tech landscape
Just some personal thoughts from development and business perspectives:
Apple - iPhones and iOS; top of the line industrial design and beautiful user interfaces. Combine this with the maturity of the Mac OS platform, excellent native development tools and users willing to buy apps on AppStore this is a top platform to target. The requirement for Macs & to learn Objective-C are initial Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com1tag:blogger.com,1999:blog-5988180.post-32407082780315852002009-11-16T08:16:00.000-08:002019-02-06T21:15:50.818-08:00MBET - what do you learn in an MBA?
1) People don't often invest in ideas, people invest in people. eg. Heck, we are going to have fun together building 'x'.
2) Don't just create cool technology, instead produce something that solve a customer pain. hint: If they don't want to pay for it, then it is probably just a cool toy.
3) What you decide to not do is sometimes more important that what you decide to do.
4) Focus on making Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com2tag:blogger.com,1999:blog-5988180.post-40375597489940026702008-10-28T18:01:00.000-07:002019-02-06T21:17:21.550-08:00Windows Azure - Microsoft's cloud platform
Microsoft doesn't seem to have changed it's strategy a little bit even though many have hailed this as a 180 degree turnaround. It is again copying what other startups and market creators have done (cloud computing, internet as a platform etc) and then executing on the plan so well that it is cheap(?), superior and accessible to everyone. At the same time, it also tries to win back the Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com1tag:blogger.com,1999:blog-5988180.post-17698447635666692302008-09-18T12:12:00.000-07:002008-09-19T19:35:23.052-07:00Thought Networking (from Primal Fusion)This may be unknown to anyone outside Waterloo, but yesterday a local startup (Primal Fusion) which was in stealth mode for over 3 years talked about what they are doing for the first time. It was interesting to watch the understated local hype since it was run by Yvan Couture, who had succesfully sold his startup to Agfa for a good profit and is also quite Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com4tag:blogger.com,1999:blog-5988180.post-78821574223469061652008-07-18T07:32:00.000-07:002008-07-18T07:36:59.399-07:00Product quality - Dev & QA synergy.Quality is a problem which every software development team has to deal with.We have the usual bell curves etc which we use to gain confidence regarding overall stability before we ship.But, here are some observations based on the timelines of the curve & potential dev/qa synergy.There are some projects where many bugs are found after the 'code freeze'. This is aVishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-91071830350383153622008-06-10T20:09:00.000-07:002008-06-10T20:20:04.273-07:00mobileMe - Wow!The sleeper announcement at the Apple WWDC was probably mobileMe. It is what MSFT, GOOGL etc. would have loved to build. I had earlier blog entries indicating that I liked .mac & how synchronization is the key to merge the web & devices (eg. SharpCast). Microsoft tried it with LiveMesh but failed before it even took off.(To be fair, I never tried the service) The seamless Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-85474532454589712782008-04-14T20:06:00.000-07:002008-04-14T20:29:41.763-07:00Google App Engine - First impressions* It is free, but with quotas. The business model is probably to get others to build apps so that Google can get users, searchable data & then eventually offer a business edition.* You don't have to worry about hosting, load balancing & scaling which is indeed huge since writing the web application is the easiest among all of the above.* The initial release isVishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-54694289463665893272008-03-13T11:52:00.000-07:002008-03-13T11:55:24.139-07:00iPhone SDK - First impressions.This is probably the most comprehensive SDK released for a mobile OS, even though Windows Mobile probably is similar. It must have been easy for Apple since most of the functionality was already in in Mac OS, throw in a different UI framework & a bunch of innovations on touch, acclerometer and location sensing you have the mobile SDK. RIM's blackberry SDK is many Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-60151450458789338952007-12-16T17:27:00.000-08:002007-12-16T17:34:53.260-08:00Parallels vs VMWareJust a mental note to myself to keep off Parallels on Mac if I have a choice, it wasted almost my entire weekend since it had trouble after I upgraded the OS to Leopard & installed new parallels trial. XP image kept on crashing after a Windows update. Finally, I switched to VMWare and had to reinstall all OSes. Hopefully, it is just a problem with the trial copy, but I found Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-75217311714228076722007-10-09T15:58:00.000-07:002007-12-16T17:33:53.614-08:00Ruby on RailsIt is hard to be excited over a new language nowadays, but recently I came across one which seemed to make web & database development fun again.I am referring to the combination of Ruby & Rails (RoR). This was a framework built for real world development problems rather than for 'building a framework' as per it's founder.It has several interesting features: 1. Lack of strong typing &Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-742526524900120332007-08-19T10:36:00.001-07:002007-08-19T11:11:48.148-07:00The end of SOAP webservices.It is probably a bit early to say this, but SOAP webservices doesn't have a future.I had been a myopic fan of SOAP webservices and one of the early adopters of the technology in various implementations - .NET 1.0, axis, xfire, JAX-WS, BPEL, JSR 180 etc. It is a great example of how excitement about a new technology can blind your judgment.It failed the single most Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-57381781790566101362007-07-25T05:18:00.000-07:002007-07-25T05:34:56.191-07:00Future of the web.The answer is right in front of you even if you don't see it already:The web started with simple HTML.Then came the dynamic web where content is dynamically generated for the users.Then came the wikis which enabled editable webs.Then web applications got more interactive with AJAX.The next level of interactivity is where applications execute and behave like Desktop Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-70033574881493800832007-04-28T07:58:00.003-07:002007-04-28T07:58:51.212-07:00Importance of "keeping it simple".Sometimes you have to keep repeating a message so that everyone gets it, at the risk of sounding like a broken record. There was a quote in this month's Popular science which applies to software world as I had been trying to say in earlier posts- "good engineers find solutions to hard problems that are complex and hard to understand, but great engineers find solutions to hard problems Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com2tag:blogger.com,1999:blog-5988180.post-3560812458967434212007-04-10T19:07:00.000-07:002007-04-10T19:09:56.322-07:00ABCs of good designGood design and architecture has always been identified as a key requirement for a product to succeed in the long term.But how how can you measure the quality of a design or identify the recipe to come up with the best design always?These are not questions with easy answers, but help to illustrate the challenges in the area.There have been many approaches to improve the qualityVishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-28188947611369769902007-04-04T13:52:00.001-07:002007-04-04T14:06:36.635-07:00OS ReviewI have always been a Windows admirer, but a newly converted Mac fan, so it is probably time to see the OS world in a more 'objective' way! Today Linux It is great as a server - controlled environment and free software without support works just fine. It is not a good home machine OS - at least not yet for the typical person who need driver support for his peripherals and who Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-82226189911963938592007-03-11T17:59:00.001-07:002007-03-21T17:27:22.117-07:00ABCs of product developmentThese should be obvious to anyone who has developed a real software product before. But, recently I had talked to a few people who were working on their first large scale projects on their own but were making some basic mistakes without realizing it. 1. If you are developing a new product or technology, prototyping and research is a must. Agile development methodologies are good, but for a largeVishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com2tag:blogger.com,1999:blog-5988180.post-48666156257933430942007-02-10T06:34:00.000-08:002007-02-10T06:58:03.907-08:00Story of a website being hackedHosting a website at home is one thing, but hosting one on the live internet is a whole different ballgame. Yesterday, I decided to expose one of my Xen virtual machines at home to the internet just to see if it works. I wasn't expecting to be hacked in less than an hour though with just 2 ports open. It brought down my jboss server(did a security no-no by running Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-43189376950475587992007-01-07T09:58:00.000-08:002007-01-07T10:10:55.462-08:00Customers & managing complexityThe only reason you should ever be solving a complex problem is if it makes your customer's life so much more simpler. But again, all complexity can be put into this category, so think twice before you do that!Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0tag:blogger.com,1999:blog-5988180.post-69809838290487111672006-12-06T12:47:00.000-08:002006-12-06T12:48:11.608-08:00Finding the simpler problems to solve and focusing on the right things.This is one of those common sense things where most big corporations and even many smart people go wrong. It is not good enough if you found a tough problem and then solved it. Many smart people often fall in this trap because they love solving tough problems.It is more important to find the simpler problem and then solve it. Vishnu Varadarajhttp://www.blogger.com/profile/03021388439150345778noreply@blogger.com0