Perspectives from the *other side* on Software, Management and Life

Saturday, April 26, 2008

Enterprise and Web technology convergence

Bidding on new software projects is a combination of art and science. A project's bid must entice the 'buyer' both on technical and budgetary matters. It is critical to assess the mindset of the buyer/customer before making the final bid. Some customers want to be at the cutting edge of technology, while others prefer to be on the tried and tested platforms. The bid scope and cost vary significantly with the approach and hence the need to determine the customer mindset. The same applies for in-house projects where the end deliverable may have metrics other than "business enablement".

All said, the two major camps of technologies ie Enterprise and Web are converging. Web technologies are significantly cheaper to develop in, deploy and maintain. While enterprise technologies require an army of developers, licensing costs and huge maintenance funds.

Before I lay out the argument as to why the two technologies will converge in the near term, let me first state what I regard to be Enterprise technology. Enterprise technology generally consist of both software and hardware configurations and technologies inline with the following set of norms:
1. Software
a. Unix (few exception with Windows)
b. Oracle
c. Java (.net when on Windows)
d. n-tiered architecture
2. Hardware
a. Deployment on multiple machines. Multiple webservers, appservers and databases
b. Emphasis on failover and high availability
c. Load Balancers
d. Firewalls


(typical enterprise architecture)



There is a significant learning curve associated with becoming an enterprise architect ie someone with the ability to develop applications in the enterprise given the variety of technologies and fancy features available.

Web technologies (simply defined as LAMP or equivalent with RoR etc) have matured and are quickly becoming part of the mainstream. These are generally hosted in 3rd party data centers and use "pay as you go" services and infrastructure. Their biggest positive being quick time to market.

The big difference, other than the technologies being used, is the mindset of the architects and the developers. Web technology has an agile and rapid taste to it, while enterprise has a documentation centric, heavy process and cross department coordination and long planning sessions taste.

The phenomenon of these market segments being merged poses a threat to one entity. Since web technologies are the under dogs, the threat is to the enterprise segment. Enterprises now need to reassess how they'll remain competitive in a market moving towards SaaS (software as a service) and where technology costs should further go down to drive the bottom line. Newer competition may leap frog existing technology centric enterprises through the use of cheaper and quick time to market web technologies.

The convergence is imminent because:
  1. Grid computing is here to stay. This has lead to 3rd parties to build out server farms on a pay-per-use model. The business model requires this to be cheaper compared to an in-house datacenter. While specialization and economy of scale will drive the cost per unit to go down. Since web technologies are grid friendly from the onset; lightweight and single threaded execution model, they can take advantage of these immediately without the need to re-engineer an app as will be required for most enterprise apps.
  2. Barriers to entry for businesses have been reduced with globalization. With fewer barriers in place, new entrants will make use of low cost web technologies to enter markets with creative ideas. At the end of the day, the end user is concerned with utility and cost, and if newer entrants beat existing enterprises on both, the users will leave for greener pastures. However since the cost of migrating to a different system is non-trivial, existing enterprises can hold onto their customers as long as they keep the cost differential within the threshold. Since enterprise technologies are expensive to maintain and enhance, they've got an uphill battle.
  3. Opensource is mainly web centric, enabling significant R&D expenditure to be outsourced when using web technologies. While enterprise technologies still require huge licensing costs to offset the R&D costs.
  4. Web technologies now come with the backing of the industry gorillas like google, amazon and Sun to name a few.
My prediction is that enterprise technology will likely be pushed into the same market as the "mainframe" world. It will be critical, but not mainstream. A small number of niche developers will hammer at it somewhere in the corporate basements. Datacenters will disappear from corporate premises (except the big financials where data security will always be paramount).

Labels: ,

Wednesday, April 16, 2008

Increase in calling rates to Pakistan

I just read with horror about PTA increasing calling rates from outside to Pakistan. It is beyond my imagination. Reasons are as follows -
  1. Increasing international calling rates will not benefit the economy, to the contrary it will burden it. Here's why, we are a country of 160 million ...thats a huge number ie we need to play with economy of scale. Increasing the rates will reduce the call terminations into Pakistan significantly. Overall there will be a loss of revenue to PTCL as individuals hold back from calling Pakistan, reduce their call times and switch over to VoIP. By going the walmart route, they can easily increase both their top and bottom line ie reduce rates!
  2. "The sources privy to the PTA, however, argued the increase in ASR rate was unlikely to damage the revenue, as the authority had already deployed technical solutions to check grey traffic brought into the country through illegal means." Are they smoking the magic dragon? What makes PTA think they're even half as competent as they need to be. There is NO way they can stop VoIP, they'd probably break every other internet app before locking in on VoIP. Afterall, this is the authority responsible for the international outage of Youtube.
  3. Damage to Pakistan's economy. By increasing the international calling rates to Pakistan, foreign firms will likely enforce calling restrictions within their offices to keep costs under control. This will generally lead to barriers in the free flow of information, which is generally key for keeping good relations with your customers. And it may actually be a blocker for new customers who'd add another column in their partner evaluation just on "communication costs".
  4. And why is VoIP illegal in Pakistan anyway? I, along with every other individual and firm dealing with US based customer use VoIP. Since PTCL is no longer a government run monopoly, it makes no sense to feed white elephants. Majority of the Pakistanis could care less if it goes out of business tomorrow.

Since no rant is worthy of being read without a conclusion proposing a solution, here's mine. Fire ever bum in PTA ... this price hike will benefit no one, not even the morons in PTCL recommending the price hike. Incase anyone's *wondering*, PTA is run by a retired army man!

Saturday, April 12, 2008

Developer Productivity

I've read a number of articles on developer productivity, but NONE of them have helped me increase it from a perspective of a "C" level position. There is just too much organization and turfs for me to drill through to figure out if a certain team as a whole stinks, or if I'm responsible for their poor productivity by not creating and ensuring a path to success.
Here are a few articles I've read but as you'll see (if you've ever managed multiple teams), they dont come close to depicting the true picture:
(image from http://www.worth1000.com/emailthis.asp?entry=387650)

  1. http://forums.construx.com/forums/t/186.aspx
  2. http://wsdj.sys-con.com/read/43343.htm
  3. http://www.solidstategroup.com/page/1623
  4. http://fc-md.umd.edu/projects/Agile/Maurer.htm

This topic and problem is as old as software development itself. There's even a whole field around it ie software engineering with the million dollar question being how to make software development an aspect of engineering whereby building major software systems boils down to using sound principles and processes. This is in contrast to the "heroics" rule where a star developer delivers the goods while the majority of the team rides the boat as tourists.

The jury is still out on the "perfect process and principles for developing software", with agile development being the new thing. In the end, being responsible for delivery of technology projects remains a headache at best. A GOOD article on the subject of productivity is:

http://haacked.com/archive/2007/06/25/understanding-productivity-differences-between-developers.aspx

My two cents on the topic are:
1. All Developers must have an incentive to be productive. This means:
a. Use logical metrics to measure productivity ie Do not use Lines of Code as a measure etc etc, as they certainly dont hold true in today's coding world of smaller is better. Use peer feedback and be careful when getting feedback from the team manager. Word of advise on team managers - sometimes they hide their faults by throwing the blame on others or are incompetent to a level where they do not understand what productivity is.
b. Dont let poor performers bog down the good ones. If the good developers are left chasing bugs created by prolific bad coders, they will only feel frustration. In the end, the good developers will be frustrated and burnt out while the bad ones will feel on top of the world. End result - attrition of top talent!
c. Dont assign average individuals to be leads or managers. They're dead weight AND they generally frustrate top talent in the team. Get rid of them ...yes, move them out. Dont wait for a major disaster to take action.
d. Ownership. Ensure code is OWNED by developer(s). If the code base is small, the team lead can be the owner ie all checkins MUST be reviewed and approved by him or her. If the code base is large, distribute code modules amongst key individuals. Unowned code is like a rental car, there is no incentive to be "nice" and drive with care....Ops, did i let a secret out :)
e. Keep QA close to your ears. They are optimally situated to report back on the overall quality of the application. Listen to them and mentor them. Using QA resources, a process of proactively identifying careless mistakes and developers responsible for them is a key benefit. This is a key component of measuring productivity

2. Enforce tools of the trade ie Source code repository, Bug Tracking system, Project management system. The ones I'm fond of are: SVN, Trac and Basecamp. Needless to say, best practices must be followed. This works miracles when tracking down escalations.

3. Recruit only the best. Taking shortcuts when hiring in crunch is generally the excuse for having a team of mediocres....dont fall in the trap.

4. Keep in close touch with all developers. They need to know that they can always escalate concerns over their managers when they're not being heard. But keep your managers in the loop until its clear the incompetence is at the mid management level.

5. Encourage developers to be vocal. Business Analyst need to know they'll have hell coming down if they screw up - as they'll stir up a storm in a beat if development doesnt deliver no matter where the flaw is. Note of caution: Dont ever let it become a "us vs them" argument, by emphasizing objectivity and professional communication.

6. Hire or promote an excellent developer to audit coding and processes being followed in different projects after every major milestone, where possible. This individual must be a killer, someone others can learn to respect because of his or her command on logic and technology. Otherwise it's just another position and overhead everyone in the org despises (I'm not a fan of creating COE's ie centers of excellences unless youre running a large firm). If you're at an executive level position in the technology department, than you're essentially offloading some of your own responsibilities here ...but lets face it, todays business world requires executives to burn their days in meetings.

Labels: ,

Tuesday, April 08, 2008

What I've learn from failed projects

This is an open ended discussion, as a project can fail for a billion reasons. There are a number of books that discuss this topic, but I'll refrain from doing a book summary and just list a couple of takeaways from the offshored projects that I've seen fail. They are (in descending order of importance):



  1. Expectations management. Its easy to say "yes", "will do", "ok", even when its feared that meeting the committed items is going to be difficult. Not being candid with your feedback and sometimes not forcing your opinions back on customers or client PMs is a weakness. Today's aggressive business environment requires technical leads/offshore PMs to be vocal and understand the end objectives of the business through the use of corporate goal buzz words in their arguments ...you know, the fluffy statements like "customers first", "quick time to market", "Return on Investment", "Strategic advantage" ....in the end decisions get made based on how well an argument is presented rather than just on its merits.
  2. Poor project planning. The usuals ...ie tracking progress, managing scope, being on top of client emails etc etc
  3. Technical impotence. Managing a technical team has always been a pain in the .... All organizations I've consulted with or managed have had business screaming down on the IT/technical teams. So first of, come to terms with the fact that you're likely working with engineers with deep flaws (unless you work at Sun, Google etc etc). You need to ensure you setup an environment where their flaws are reduced over time and their faulty coding does not become a source of embarrassment for you. For this, an environment where they are able to flourish and grow is needed, while making sure you're able to weed out the bad. Emphasis on weeding out! By not taking out bad engineers, you are creating an dis-incentive for other engineers to work hard. If thats not a possibility, raise it with senior management so you're not seen trying to play blame games when things go south.
  4. Communication. Most offshore managers have bad english. This may not seem like a disadvantage, but if an escalation happens - you'd much rather not have an executive scratching his/her head just to figure out what you're saying. By the time you're able to get one thought across, the incompetent moron with a good grasp on the language may have fired out 10 thoughts, increasing the everprobability of having his side of the story being sold.
  5. Relationship management. Try to figure out something to have a conversation about that is outside work. F1 racing, Football, cricket etc. It makes a big difference in how you're perceived by others and their attitudes towards you (specially if they're sitting on the other side of the world). Perceptions help greatly as it allows for better flexibility, specially during crunch time.

Labels:

Sunday, April 06, 2008

Challenges of offshore development (view from the offshore side)


The US employment market is fairly vibrant with an array of talents. It is a market like no other. It's archille's heel being immigration ie when the flood gates are open, the US employment market is full of all forms of talent. When the flood gates are closed (as they have been in the last few years), its a little bit more barren.
I will not get into the benefits of offshoring, needless to say expectation must be inline with reality otherwise offshoring will be labeled a failure no matter what.


(Image taken from http://www.ogi.org.uk/images/Team-effort-to-get-the-Pont.jpg)
Returning back to offshore development and the number of challenges from the eyes of an offshore perspective, here are a couple:
  1. Extra hot market - new offshore development houses are sprouting everyday. And since talent isnt as plentiful as the number of companies, attrition rates are relatively high. In this environment, things can quickly go from bad to worse as the saying goes "when the going gets tough, the tough gets going" only in this case the developers get going ...to another software house! Hence the need to ensure escalations are kept within bounds. US based managers have lately picked on a couple of buzz phrases eg "we need to have all hands on deck on the weekend", "i need access to development 24/7", "we're all overworked but we got to do what we got to do". What such managers fail to realize is that this leads to a boiler room environment where top talent is stretched close to its breaking point.
  2. Blame game - offshoring exposes the ugly truth generally hidden away. A development process that may have been going "smooth" will go haywire with cost overruns and delays when offshored. Both parties are to blame but offshore generally gets the brunt of it as its the new element in an old recipe. A common cause is PM's being incapable of understanding the basic elements of software development. Let me qualify this, when business is done in one room, mistakes and other not-so-well-thought-out directions are quickly identified and corrected. Put offshoring in the mix and its easy to get the development team running in the wrong direction only to realize on D-day that there is a typhoon. As far as "smooth", I have yet to see a software development initiative in a fortune 500 company (all onshore) that is delivered on time and on budget. Software development is an agile business that requires constant course corrections.
  3. Local project managers - Projects that are managed locally or use a local liaison (as team lead etc) require someone who is good great at communication, managing expectations, pushing back customers/onshore, pushing hard local development and ensuring quality deliverable. This is a tall order for anyone. Finding such a PM offshore is extremely difficult, with most US based PM's being poor in a number of these skills.
  4. Quality Engineers - everyone wants a google engineer who works smart and works hard (a 120% of it ;) ).Everyone. I cannot emphasize this enough, offshore productivity is slightly lower than US productivity. And no engineer wants to work a weekend, but offshore engineers do. They are generally not paid overtime and do not have equity/stocks in the clients they work for ie ZERO incentive to work overtime.
I'll probably touch this subject again later when i get some more time. Its a subject that i'm passionate about yet is too big to discuss in one sitting :).