In the past several days I have run across a number of instances where innovation has been discussed with one central theme. Where has it all gone? I finally ran across the answer this morning.
The first discussion on the topic arose around why a very large IT product company would cut its R&D budget dramatically from one year to the next. My answer? It is cheaper to acquire technology from smaller companies than it is to fund the development in house. Not to say that this is always true but I was speculating on the real answer for this organization. I believe I am right given that the company announced a continuation of its acquisitive nature in the coming fiscal year.
Next up came a discussion around why a fairly large, long standing organization was not an ideal buyout target. The answer? The CEO has turned down a number of approaches from other organizations and is known throughout the tech industry to be difficult to get along with in this regard. It would seem that anyone approaching him about a buyout is rebuffed almost immediately.
Finally, today I saw an article announcing a new book by a former CTO of Cisco, Judy Estrin. In her book she outlines her concern about the lack of innovation in America in general, and specifically within the tech industries. Her conclusion? Short term vision on the part of senior management. The emphasis on short term gains, building companies to be taken over by larger ones after a short time, funding tech start ups to be bought by the likes of Cisco without any consideration for building anything for the long term was cited as a growing concern. It will be interesting to read the book but I agree with the answer. The situation applies in other areas as well, like oil. Using oil profits to pay bonuses and dividends rather than developing alternative energy sources to replace a depleting one.
Cheers to the tech CEO that would rather build his own organization than to sell out.
Friday, August 29, 2008
Thursday, August 28, 2008
How much more proof do we all need?
In my last post I made the statement that 80% of the time human error can be accounted for as the cause of major IT outages. This goes for enterprises, carriers and the government.
Today it was published that the cause of a major outage of the FAA NADIN network was due to "human error that "resulted in the wrong configuration data being loaded onto the switch". This is not an uncommon error. Noteworthy in the release of information was that the configuration error took place on an IPX 9000 packet switch. Anyone other than me know what that is?
Secondary to the outage was the fact that the backup provisions for NADIN to process flight plans calls for a system in Utah to pick up any load from the failed system in Atlanta. The problem was that the queue built up caused so many delays and re-inputs from the airlines that the system in Utah could not keep up. As I mentioned earlier it is wise to test backup systems to see what will happen when they are called upon to do their duty. Clearly this was not the case here. Now the FAA is talking about adding a third system to the mix for backup. How much do you want to bet that routing issues will be the next thing that will plague this system and none of the systems will pick up the load when needed?
The FAA's antiquated management and procurement practices make it hard for them to get things right. The good news is that in this outage planes were held up on the ground.
The 80% rule still applies but it can be overcome with smart planning and execution.
Today it was published that the cause of a major outage of the FAA NADIN network was due to "human error that "resulted in the wrong configuration data being loaded onto the switch". This is not an uncommon error. Noteworthy in the release of information was that the configuration error took place on an IPX 9000 packet switch. Anyone other than me know what that is?
Secondary to the outage was the fact that the backup provisions for NADIN to process flight plans calls for a system in Utah to pick up any load from the failed system in Atlanta. The problem was that the queue built up caused so many delays and re-inputs from the airlines that the system in Utah could not keep up. As I mentioned earlier it is wise to test backup systems to see what will happen when they are called upon to do their duty. Clearly this was not the case here. Now the FAA is talking about adding a third system to the mix for backup. How much do you want to bet that routing issues will be the next thing that will plague this system and none of the systems will pick up the load when needed?
The FAA's antiquated management and procurement practices make it hard for them to get things right. The good news is that in this outage planes were held up on the ground.
The 80% rule still applies but it can be overcome with smart planning and execution.
Wednesday, August 27, 2008
The Eighty Percent Rule
In the last 24 hours we have all been told of a major disruption in the air traffic control system in this country. The word is that a "communications failure" in a FAA facility in Atlanta was at fault for numerous flight delays across the country and in particular in the Eastern US. Further information indicates that a network failure between Atlanta and a facility in Utah caused the issue.
This situation reminds me of an old adage in the networking business " 80% of all network failures are caused by changes made by humans". In nearly 30 years of observations I can validate the saying. Interestingly enough one can easily prove the 80% rule by putting an embargo on network changes during a time of low staffing, say the end of the year holiday period. Year after year of doing this resulted in that period of time being the lowest instance of outages for the entire year. With no one around and no one doing any changes in the network things just ran like they were supposed to.
What does all this mean? If there is a period of critical operation then minimize changes to your environment during that period. The last thing you want to do, say if your are a candy maker, is to do a major system change during the period leading up to Halloween (Hershey did an ERP upgrade during this time and did not produce enough candy to meet the Halloween demand and thus had their worst performing quarter in their history).
My guess is that we will find out that the recent FAA outage was a direct result of a network change that went in either untested or poorly tested at best. The other possibility is the lack of testing of redundant links so that you know they work when needed.
Stay tuned on this one. I have an 80% chance of being right again.
This situation reminds me of an old adage in the networking business " 80% of all network failures are caused by changes made by humans". In nearly 30 years of observations I can validate the saying. Interestingly enough one can easily prove the 80% rule by putting an embargo on network changes during a time of low staffing, say the end of the year holiday period. Year after year of doing this resulted in that period of time being the lowest instance of outages for the entire year. With no one around and no one doing any changes in the network things just ran like they were supposed to.
What does all this mean? If there is a period of critical operation then minimize changes to your environment during that period. The last thing you want to do, say if your are a candy maker, is to do a major system change during the period leading up to Halloween (Hershey did an ERP upgrade during this time and did not produce enough candy to meet the Halloween demand and thus had their worst performing quarter in their history).
My guess is that we will find out that the recent FAA outage was a direct result of a network change that went in either untested or poorly tested at best. The other possibility is the lack of testing of redundant links so that you know they work when needed.
Stay tuned on this one. I have an 80% chance of being right again.
Monday, August 18, 2008
Management Challenge-Part time employees
This one should be easy but it turns out it is not.
The situation is this- you have someone who, for any number of reasons, wants to work part time. You like the person and they have proven to be a valuable employee. Perhaps the person is new to you and you like what you have seen thus far but a full time arrangement is out of the question. Do you agree to a part time arrangement?
The challenge is multifaceted. First, you could get a good am out of productivity out of this person even though you do not have them full time. They like what they do, they are energetic but have other responsibilities. Perhaps they are willing to take a pay reduction in order to facilitate the arrangement. If you agree you will make them happy and get a good employee as a result. The bad news is you may build up resentment on the part of other members of your staff. "Why does she/he get to work part time and I don't?" might be something you hear. As a practical matter you can ill afford to run your group or operation with a whole staff of part time employees. The other issue turns out to be an accounting/HR one. A name on an org chart typically counts as a head count whether they are full time or part time. When it comes time to "adjust" headcount you will be faced with the issue of cutting a Full time or Part time person. It will come down to a question of productivity at that point. It is sad to say but there are situations where you might be getting more from a part time staff member than you do one that is full time.
Obviously there is no clear cut answer in this situation but it is an interesting situation and one that will require all of your skills in management and diplomacy to work through when the time comes. With the population aging as it is, creating more need for caregivers, you may be dealing with this sooner than you think.
The situation is this- you have someone who, for any number of reasons, wants to work part time. You like the person and they have proven to be a valuable employee. Perhaps the person is new to you and you like what you have seen thus far but a full time arrangement is out of the question. Do you agree to a part time arrangement?
The challenge is multifaceted. First, you could get a good am out of productivity out of this person even though you do not have them full time. They like what they do, they are energetic but have other responsibilities. Perhaps they are willing to take a pay reduction in order to facilitate the arrangement. If you agree you will make them happy and get a good employee as a result. The bad news is you may build up resentment on the part of other members of your staff. "Why does she/he get to work part time and I don't?" might be something you hear. As a practical matter you can ill afford to run your group or operation with a whole staff of part time employees. The other issue turns out to be an accounting/HR one. A name on an org chart typically counts as a head count whether they are full time or part time. When it comes time to "adjust" headcount you will be faced with the issue of cutting a Full time or Part time person. It will come down to a question of productivity at that point. It is sad to say but there are situations where you might be getting more from a part time staff member than you do one that is full time.
Obviously there is no clear cut answer in this situation but it is an interesting situation and one that will require all of your skills in management and diplomacy to work through when the time comes. With the population aging as it is, creating more need for caregivers, you may be dealing with this sooner than you think.
Thursday, July 31, 2008
Internet traffic jam?
I came across an article recently by one of the pioneers of the Internet, Larry Roberts, for whom I worked many years ago. Larry's article was insightful and pointed out that there are some reasons to believe that there should be classes of service for those accessing the Internet. Larry is a visionary in network protocols and has started yet another provider organization, this time to provide Internet traffic carriers with the ability to throttle traffic "flows" based on class of service.
In reading the article it struck me as odd that Larry would advocate using bit rate to determine class of service and therefore the fees associated with Internet access. I have a relatively slow broadband access rate in my home office and pay a modest rate for the service I receive. Power users on the other hand might pay a larger fee for their higher grade of service. My thought was that I would rather pay for the grade of service the information I requested gets from my ISP rather than just the bit rate I am able to get from the carrier. This "Contracted Information Rate" would allow agreement between me and my service provider on what grade of service they would be willing to provide as measured over a set period of time. This would allow me to be assured of receiving a set service level regardless of the bit rate of my access link, realizing that there would be a limit to the service I could receive if I continue to have a relatively slow access path. Measuring this access would be relatively easy over a longer period of time rather than just complaining about not achieving peak bit rates on my link over a very short span of time.
It is very clear that the more bandwidth demanding applications that are developed and delivered over the Internet the more likely we are to have occasional slowdowns. The service providers have every right to charge for the grade of service delivered to end users but at the same time the end users have a right to receive a quality of service that is appropriate to their use and applications. Restricting rate flows is OK as long as it is being done in a way that achieves the agreed to objectives of both parties involved. Also, I do not want to pay for poor performance from a carrier based on bit rates when what I really care about is getting or transmitting the information I am interested in. It is also clear that we can not expect to be able to continue to simply "fill the pipe" with information and expect performance to be at link speeds at all times. Networks do not work that way in the long run. Maximize the utilization of the network asset, yes, but do not expect to fill it 24/7 and achieve any acceptable performance.
This may all seem to be an argument over the same thing but I feel that there is a very fine difference. The technology solution that offers a way for me to send or receive information within a certain time frame "absolutely, positively" will win my business and ultimately allow everyone involved to win as well.
In reading the article it struck me as odd that Larry would advocate using bit rate to determine class of service and therefore the fees associated with Internet access. I have a relatively slow broadband access rate in my home office and pay a modest rate for the service I receive. Power users on the other hand might pay a larger fee for their higher grade of service. My thought was that I would rather pay for the grade of service the information I requested gets from my ISP rather than just the bit rate I am able to get from the carrier. This "Contracted Information Rate" would allow agreement between me and my service provider on what grade of service they would be willing to provide as measured over a set period of time. This would allow me to be assured of receiving a set service level regardless of the bit rate of my access link, realizing that there would be a limit to the service I could receive if I continue to have a relatively slow access path. Measuring this access would be relatively easy over a longer period of time rather than just complaining about not achieving peak bit rates on my link over a very short span of time.
It is very clear that the more bandwidth demanding applications that are developed and delivered over the Internet the more likely we are to have occasional slowdowns. The service providers have every right to charge for the grade of service delivered to end users but at the same time the end users have a right to receive a quality of service that is appropriate to their use and applications. Restricting rate flows is OK as long as it is being done in a way that achieves the agreed to objectives of both parties involved. Also, I do not want to pay for poor performance from a carrier based on bit rates when what I really care about is getting or transmitting the information I am interested in. It is also clear that we can not expect to be able to continue to simply "fill the pipe" with information and expect performance to be at link speeds at all times. Networks do not work that way in the long run. Maximize the utilization of the network asset, yes, but do not expect to fill it 24/7 and achieve any acceptable performance.
This may all seem to be an argument over the same thing but I feel that there is a very fine difference. The technology solution that offers a way for me to send or receive information within a certain time frame "absolutely, positively" will win my business and ultimately allow everyone involved to win as well.
Monday, July 21, 2008
Network Administrator gone bad-Told you so
Back on May 20th I posed the question whether network administrators, or anyone with Admin rights to a resource for that matter, should be licensed. The premise was that if the person is a professional then they should be willing to submit to a background check, and to operate under a code of ethics in their daily work.
As of last week we have now seen the worst case scenario play out. A network admin for the City of San Francisco created a super user for himself for the network resources and locked out the other admins from the network. He then proceeded to hold the network hostage in order to ensure that his employer would not take action on a performance compliant they were building. What information that is being released now makes this story even more unbelievable.
It would appear that the city of San Francisco hired a person previously convicted of aggravated burglary to maintain part of the city's IT infrastructure with the full knowledge of this past criminal history. He is now sitting in jail and the city still does not have it's network back in full operation. Under the concept risk management and good security practices this hire would not have been viewed as a wise move. Under the previously suggested practice of licensing system administrators this person would not have been allowed to have admin rights at all. All that said, who was watching the store on this one? Were there no configuration audits during this person's tenure in this ill advised position? How about automating that audit function? Immediate notification of an unauthorized change to the network environment would have been advised don't you think?
This type of transgression should have never taken place and was very easily prevented. Answer-if a hiring decision seems like a bad idea, it probably is. In this case, I am 100% sure it was a bad idea. No license for this guy.
As of last week we have now seen the worst case scenario play out. A network admin for the City of San Francisco created a super user for himself for the network resources and locked out the other admins from the network. He then proceeded to hold the network hostage in order to ensure that his employer would not take action on a performance compliant they were building. What information that is being released now makes this story even more unbelievable.
It would appear that the city of San Francisco hired a person previously convicted of aggravated burglary to maintain part of the city's IT infrastructure with the full knowledge of this past criminal history. He is now sitting in jail and the city still does not have it's network back in full operation. Under the concept risk management and good security practices this hire would not have been viewed as a wise move. Under the previously suggested practice of licensing system administrators this person would not have been allowed to have admin rights at all. All that said, who was watching the store on this one? Were there no configuration audits during this person's tenure in this ill advised position? How about automating that audit function? Immediate notification of an unauthorized change to the network environment would have been advised don't you think?
This type of transgression should have never taken place and was very easily prevented. Answer-if a hiring decision seems like a bad idea, it probably is. In this case, I am 100% sure it was a bad idea. No license for this guy.
Saturday, July 5, 2008
IT Security Spending- Is it enough?
On average an enterprise organization spends anywhere from 3% to 6% of its IT budget on security infrastructure. This budget amount is divided between operating expeses (nearly 70% of an IT budget which just keeps the wheels on) and capital expenditures intended to improve things and meet new user requirements (making up the other 30%). Is it enough?
I would say that it depends upon the organization and the vaule of the information assets the enterprise is looking to protect. A good example is a financial services firm. This organization is entrusted with information that could litterally change the course of the lives of millions of people should it fall victim to a security breach. In this instance I would hope that such an organization was devoting more than say 5% to keep my money safe. Speaking of which I got a phone call recently from a rather large financial services organization indicating that an account I have with them was subject to an attempted fraudulent act. All was well and I was pleased with the proactive nature of the call. I make it a point to thank the caller every time even though I realize they may be trying to save their company pain and financial loss more than they are trying to save me.
Just another example of how it all depends upon the vaule of the asset and the organization's tolerance for risk,
I would say that it depends upon the organization and the vaule of the information assets the enterprise is looking to protect. A good example is a financial services firm. This organization is entrusted with information that could litterally change the course of the lives of millions of people should it fall victim to a security breach. In this instance I would hope that such an organization was devoting more than say 5% to keep my money safe. Speaking of which I got a phone call recently from a rather large financial services organization indicating that an account I have with them was subject to an attempted fraudulent act. All was well and I was pleased with the proactive nature of the call. I make it a point to thank the caller every time even though I realize they may be trying to save their company pain and financial loss more than they are trying to save me.
Just another example of how it all depends upon the vaule of the asset and the organization's tolerance for risk,
Subscribe to:
Posts (Atom)