in IT/AV Report, Spring 2007
Brave New World
By Neal Weinstock
How cheap, great quality encoders change everything. Maybe.
You’ve heard about MPEG4 AVC, otherwise known by its ITU-R standard designation of H.264. Probably heard about it ad nauseam, and are still left to wonder how this significant improvement in video compression makes any real difference to your business other than the usual, “now things go a little faster?”
And it’s true, in the great scheme of things, H.264 is only a little faster than the older standard, MPEG2. It brings a roughly 50% improvement in efficiency, depending on the bit rate. (At higher bit rates and video quality levels, H.264 offers only a tiny improvement, or none at all, over MPEG2; at low bit rates typical of internet videoconferencing or distribution, the improvement can be close to 100%.) For a technological advance that has been in development for a decade, this is not spectacular.
Compare it to the advances in CPU power and memory storage over the same period, both more than a hundredfold.
Hardware vs. Software
Most of why these other technologies have advanced further is that they involve hardware. Chips can be built bigger, with smaller connection distances between circuits, and with faster clock rates, all of which (put very simply) increases the speed at which they run software. Video codecs have been, and will continue to be, prime beneficiaries of the gains in chip power and memory. That’s because, generally speaking, video is the fattest payload in wide use throughout the digital domain. We’re able to take advantage of these codecs because our digital devices run ever faster, and locate storage of video at various places across the network.
Write software into specialized hardware, however, and you make it run even faster. Once that software becomes “firmware,” or blocks of code embedded into chips, the cheaper/faster/stronger improvement curve accelerates. MPEG2 encoding and decoding got onto this accelerated hardware portion of the curve long ago. So did MPEG4 decoding. So what’s been holding up MPEG4 encoding?
In a word, the standard hasn’t been finished. Indeed, the H.264 standard may never be finished! Write an unfinished standard into firmware and, when some changes in it become accepted by the standards-making body, that firmware may become obsolete. It costs a lot of money to design and build chips: a lot more than it costs to write code as software and run it on generic hardware.
On the other hand, that spending on hardware is almost all fixed cost, so that, if you have a huge market, the same code eventually will be able to sell more cheaply as hardware than as software. But, if a standard changes while your code is only software, you can patch the software, often can do online updates to all registered users, with the only people left in the lurch being any software pirates and their duped customers. If a standard changes when your code is already hardware, you can throw out all the hardware.
OK. So writing H.264 into hardware while it’s still an unfinished standard is a bad idea. So why is it good that companies are now doing exactly that?
It is good because H.264 has now reached the mostly finished (for the time being) stage at which a whole lot of other standards also reside. Ethernet, for example, or Wi-Fi. Ethernet was conceived decades ago, but it’s still not “finished.” There will be more forthcoming iterations of that standard for many years to come. Most, however, promise several years of stability, enough time for us all to get lots of use out of compliant hardware, and for the companies that make that hardware to make some money on it.
H.264 has reached that point, finally, which is a little paradoxical because H.264 still cannot be used to accomplish the main function it was supposed to do originally.
MPEG4 was first thought up as a standard that would combine multimedia with streaming. It was the first video compression standard conceptualized after the rise of the public internet, and also of interactive TV (which flopped in the US but achieved wide success in several other countries). So, naturally, its first proponents thought about combining compression with capabilities to hyperlink objects in the frame, both within and outside of video, embedding objects in video, and so forth. None of that happened, probably because it’s all pretty complex stuff.
Many of the companies with employees taking part in the standards deliberations have had different ideas about how these multimedia features would be accomplished; thus, the discussions have gone on endlessly until now. Also, both the IP standard for the internet and the MHEG standard for interactive TV allow other programs and protocols to supply most of the multimedia interactivity that was contemplated originally as part of MPEG4. Not all of it, by any means, but enough to reduce the clear need to come to a decision about how this stuff would be implemented within a video compression standard.
So, MPEG4, eventually designated as H.264 when it came under the auspices of the I-TUT (International Telecommunications Union Standardization Sector), came to be merely a compression standard so far, not really a multimedia standard. Multimedia standards are, however, highly likely eventual extensions of what we currently think of as the standard. Just as Ethernet has grown to include POE (power-over-Ethernet), GE (Gigabit Ethernet), switching standards, etc., so H.264 is likely to branch into a family of standards. These likely will have to do with carriage of interactive multimedia.
What’s So French About This?
There seems to be something French about this concept. Ambarella, a California company co-founded by Frenchman Didier LeGall (who also founded the greatest success story in MPEG2 encoding, C-Cube Microsystems, now part of LSI Logic), claimed in January to have come out with the first single-chip HD, real-time H.264 encoder. It is used by professional encoding market leader Harmonic and also by a number of consumer electronics companies, including Samsung.
Thomson, based in France, also now claims to offer the first true, single-chip, high-definition, real-time H.264 encoder, its Mustang chip. MnD Semiconductors (another French company) is a small startup that has delivered an interesting chip architecture capable of supporting real-time H.264 encoding in HD; it’s a multiprocessor design, but using multiple standard processors (Sun’s SPARC) that are small and inexpensive. Yet another French company, Ateme, has a single-chip, standard-definition H.264 encoder, and a four-chip HD encoder.
Je ne sais quoi
The opportunity they create, however, is a language we all speak. The main purpose of H.264 encoding right now is (as in the Harmonic application mentioned earlier) to get video squeezed into multichannel satellite or cable TV systems. Typically, only decoders have gone into consumer set-tops on the other end of the signal chain. But there are also plenty of encoders in consumer devices: video cameras, including those in mobile phones. Whenever you want to take an analog video signal, or a digital signal in a different format than the playout device uses, and play it on a digital device, such as a PC or virtually all flat-panel or projection displays, it also has to be encoded.
This is to speak of the big bucks arena of the broadcast-to-consumer and consumer-generated video signal chains. But a look at how these applications play out in terms of technology deployment can suggest some huge commercial AV implications.
TV on the PC
Elgato Systems (which is German), for example, has put a single-chip H.264 encoder into a USB stick. Its software, also on the stick, allows users to tune in TV signals on their PCs. It makes several models of this sort of product; EyeTV Hybrid lets you play video games from a game console or TV channels on the PC. It costs $150. You used to require a much more expensive card or a breakout box to do all of this, and use lots of the PC’s CPU power and RAM, too. Now it all can go on a USB stick, and all the processing is on the stick. It is, therefore, far less expensive to bring TV playout into the PC, and the PC can be used for other purposes simultaneously.
For commercial AV, this means users will be able to bring a high-quality video on a memory-stick-like device to any PC (even a very inexpensive one) and play it out in high quality on a big screen. It also means that you can bring in broadcast simply to any PC and mix it with banners and overlays at high quality on a low-end PC, and thus put out digital signage inexpensively over TV signals.
Similarly, PCs use software codecs running on CPU and RAM resources to play all video, whether in QuickTime (which forms the basis of the H.264 video format), Windows Media (which is much like H.264, so many H.264 encoders and decoders will be able to handle it, but many won’t) or other codecs (Divx, again, very close to pure H.264, Real Player, etc.). This isn’t terribly efficient but, as long as both the Windows world and Apple have done so, there has been no competitive pressure to move to the more efficient playout that hardware encoding would entail.
In fact, the incentives have all been the other way: If a PC plays video slowly and with glitches, a consumer likely will buy a new PC with faster CPUs and more RAM to play it better. Conversely, it costs a few bucks to put in hardware encoding, and every penny counts in building PCs in a highly competitive environment.
But single-chip H.264 encoding can break that logjam, and not in Microsoft’s favor. A single-chip encoder will sell really cheaply in quantity, because the goal will be to get chips eventually into every cell phone. That probably translates to pricing at under $10 this year. This adds some cost to the PC, but not a tremendous amount. For the Macintosh, which usually is priced higher than Windows PCs, the additional cost could bring real bragging rights in video play. As of this writing (in mid-April), indeed, it has been rumored that the next generation of Macintoshes will include hardware H.264 encoding.
As Windows PC-makers also incorporate these chips into their boxes, it will either increase use of H.264 or the codecs that are more compatible with it, or force Microsoft to make Windows Media more compatible with it.
Cell phones and, eventually, iPods likely will get chips that perform only low-bandwidth subsets of H.264: the most efficient bitrates, which won’t look very good on a big screen. But other consumer devices, especially standalone video cameras, PCs, TVs and set-tops, will require encoders with multiple bitrates, including higher ones that look great on big screens. It’s a brave new world, and the services that can be integrated from these inexpensive video components are limited only by the user’s imagination.