from The Web Host Industry Review

October 12, 2001 -- (WEB HOST INDUSTRY REVIEW) -- Keynote Systems Inc. (keynote.com), a leader in Internet performance measurement, released a report on Thursday, detailing the results of its evaluation of Wednesday, October 10’s live U2 concert Webcast. The event rated 3.8 on the arbiter’s scale, which ranges from 0 to 10.

The company says its measurement of the performance and quality of the event reflects the experience of users in the US and Europe, and includes some results from London and Munich.

Keynote’s performance score is built on two scales. The first is the Streaming Quality Scale, which ranks the quality of a stream from 0 to 10 based on delivery and encoding factors like redirect time, initial buffer time, late, lost and dropped packets, average frames per second and frames dropped and lost, among others. The second scale is the Rendering Score, which delivers a percentage score by measuring actual playback quality against the intended quality.

On the Streaming Quality Scale, the stream for users in the US ranked an average of 3.54 between 6:00 and 9:00pm (Pacific time), measured at an encoded bandwidth of 56Kbps. According to Keynote, availability was high, mostly between 96%-100%, for most of this time, but dropped as low as 89.9% at the start, giving it an overall average of 94.2% availability over the three hours.

In Europe, says Keynote, users experienced a similar level of quality, but at a reduced availability. And in London, overall stream quality was 3.62, the rendering score was 96.0% and availability was 88.9%. The stream was 100% available most of the time, but dropped to 66.7% during two 15-minute periods. In Munich, overall stream quality was 3.64 and rendering score was 97.0%, while availability was 86.1%. Again, the stream was 100% available most of the time, but dropped to 66.7% availability about one third of the time.

Keynote says that overall, for a low bandwidth 56Kbps stream, the U2 webcast showed good performance and availability. Its 97% rendering score means that the event was received largely as the operators intended.

The overall score of 3.54 for the U2 concert, by comparison, was higher than the score for the webcast debut of the movie “Tomb Raider,” which ranged from 3.11 to 3.22 at 56Kbps. U2 also beat out the March Madness games, which scored between 0.24 and 2.26 for streams at the slightly higher bandwidth of 80Kbps.

But higher bandwidth streams will likely rate higher, as long as they meet with their potential, says Keynote. For example, press conferences webcast for this year’s Super Bowl rated as high as 5.78 for 300Kbps streams.

“The U2 webcast did very well for the bandwidth at which it was encoded,” says Leslie Gaillard, product manager for Keynote’s Streaming Perspective service. “A score of 3.54 on a scale from 0 to 10 may seem low. And it is. But this webcast did well under the limitations of current technology.

“There are many considerations in delivering streams to a huge worldwide audience,” says Gaillard. “Undoubtedly, in order to successfully meet the demand, this stream was encoded at a low bandwidth, which delivered the video in a very small frame. Stream quality will be restricted by the lowest common denominator as long as millions of users are still on low bandwidth connections.”