Getting Ready for TechED Europe 2014

Next week I will be in Barcelona, talking about all the cool stuff we are doing with virtualization in the next release of Windows Server.  Here are my session details:

Title: CDP-B225 Software Defined Compute in the Next Release of Windows Server Hyper-V
Time: Tuesday, October 28 1:30 PM – 2:45 PM Room: Hall 8.0 Room E1
Description: Join us for a bird’s-eye view of software defined compute as we enter the next release of Windows Server. See how Microsoft envisions using Hyper-V as it relates to our overarching vision of software defined compute.

Title: CDP-B246 Sneak Peek into the Next Release of Windows Server Hyper-V
Time: Wednesday, October 29 8:30 AM – 9:45 AM Room: Hall 8.0 Room B4
Description: The technical preview for the next release of Windows Server has been released, but what can you do with it? Why do you care? Come to this session to get the nitty gritty details of all the new stuff that is in Hyper-V; learn about how you can play with it “hands on” today; and gain insight into where we are heading.

I will also be spending a lot of time on the tradeshow floor chatting to people.  So if you are going to TechED Europe this year – be sure to come by and say hello!

Cheers,
Ben

Excel Power Map October Update

Last month we announced two exciting new features for Power Map, including support for Filtering, and the ability to plot data on top of Custom Maps.

This month we’ve focused on providing quality improvements. We’ve made more fixes in Power Map than in any monthly release so far spanning filtering, custom maps, timeline, tour playback, accessibility, look and feel, 2D charts, and stability. Please continue to provide your feedback and report any errors by clicking the  button in your own Power Map tour in Excel.

Power Map is part of several analytics and visualization features in Excel, more info at Power BI for Office 365. Power Map is available to all customers with an Office 365 subscription. If you have automatic updates enabled for your Office 365 subscription, you should receive this update soon. If you don’t have automatic updates enabled in Excel, go to File > Account > Update Options > Update Now.

PowerShell DSC Does Exchange!

One of the more complex products to deploy and configure has just been made easier with the release of the xExchange PowerShell DSC Resource Module.  

Mike Hendrickson, the creator of these resources, has begun documenting how to use PowerShell DSC to deploy & configure Exchange in the article Introducing the xExchange Module for Powershell Desired State Configuration.

The xExchange module is available via the DSC Resource Kit download page, and by using Install-Module (if you have WMF 5 installed).  This set of resources is in direct response to requests from customers like yourselves. If you are working to deploy and maintain Exchange 2013, these resources can be used to solve real deployment issues, and demonstrate how to solve others, by leveraging PowerShell Desired State Configuration.

 

Check it out – Introducing the xExchange Module for Powershell Desired State Configuration

      The PowerShell Team

Compressing the Web

Be succinct.

Virtually any network-based application can be made faster by optimizing the number of bytes transferred across the network. Taking advantage of caching is a great way to minimize transfer sizes, but just as important is to reduce the size of the resources you transfer.

Data compression is used throughout the protocols and formats used by browsers, and today’s post is a summary of where you can find compression in use and how you can optimize it to improve the performance of your sites and services.

Compression Categories

Conceptually, compression is simple: recognize patterns in data and reduce repetition to minimize the size of the data. There are two major categories of compression: lossy and lossless.

When information is compressed with lossy compression, reversing the process (decompression) doesn’t result in the original data, but instead a surrogate which resembles the original either closely or loosely depending on the quality of the compression. In browsers, the primary uses of lossy compression are in the JPEG image file format, the MP3 audio file format, and the MPEG video file format. These formats utilize understanding of human perception to drop details that are unlikely to be noticed by the viewer/listener, resulting in often-huge savings in data size. While important, this post won’t discuss lossy compression any further.

In contrast, decompressing data that was losslessly compressed results in an exact copy of the original data—every byte is identical. Lossless compression is used throughout the web platform, in both the HTTP layer and internally within many web file formats including PNG, PDF, SWF, WOFF, and many more. The remainder of this post explores the use and optimization of lossless compression.

HTTP Headers

HTTP requests advertise the decompression algorithms the client supports using the Accept-Encoding request header.

Servers indicate which compression algorithm was used for a given response using the Content-Encoding response header. Unfortunately, even the updated HTTP/1.1 specification implies that servers can use compression-related tokens in the Transfer-Encoding response header, but indicating compression using the Transfer-Encoding header does not work in any major browser (all support only the chunked token).

If the server utilizes compression, it should generally include a Vary: Accept-Encoding response header to help ensure that a cache does not mistakenly serve a compressed response to a client that cannot understand it.

Algorithms

The most popular compression algorithm in use on the web is the DEFLATE algorithm, specified in RFC 1951. DEFLATE combines the LZ77 algorithm with Huffman encoding; it is straightforward to implement and effectively compresses a wide variety of data types.

In browsers, the most obvious use of the DEFLATE algorithm is for HTTP/1.1 Content-Encoding. DEFLATE is the algorithm underlying two of the three encodings defined in the HTTP specification (“Content-Encoding: gzip” and “Content-Encoding: deflate”).

DEFLATE vs. GZIP for Content-Encoding?

Given that virtually all clients allow both “Content-Encoding: gzip” and “Content-Encoding: deflate”, which should you use?

You’ll find conflicting opinions on the topic, many of which are written without the understanding that both encodings are based on exactly the same algorithm, and the encodings differ only in the header and trailer bytes that wrap the compressed data.

The GZIP encoding of a resource starts with two magic bytes 0x1F 0x8B, a byte representing the compression method (0×08, indicating DEFLATE), seven additional bytes of metadata, several optional fields (rarely used), and then the DEFLATE-compressed bytes. After the compressed data, a 32bit CRC-32 and 32bit original datasize field complete the format.

Despite its name, the DEFLATE encoding of a resource is specified to be the ZLIB data format. This format consists of two header bytes; the first contains the compression method and window-size information (the low bits of the byte are 0×08, indicating DEFLATE). The second header byte is a flag byte which indicates the compression strategy (minimize size vs. maximize speed) and serves as a checksum of the header bytes. The DEFLATE-compressed bytes follow. After the compressed data, a 32-bit ADLER32 checksum field completes the format.

So, all other things being equal, a C-E:GZIP-encoded resource will be exactly 11 bytes bigger than a C-E:DEFLATE-encoded resource, due to the seven additional header bytes and four additional trailer bytes. In practice, however, your compression program may generate wildly different results.

Now, note that I said the encoding is specified to be the ZLIB data format. Here’s what you’ll see in Internet Explorer if you try to load a page which has the ZLIB-wrapper around the DEFLATE data:

image

The problem is that Internet Explorer expects a bare DEFLATE stream without the ZLIB wrapper bytes. So, does that mean you can’t use Content-Encoding: deflate? Not really, no. In addition to supporting the proper ZLIB-wrapped data, Chrome, Firefox, Safari, and Opera all support bare DEFLATE.

Some folks recommend you avoid C-E:Deflate and just stick with C-E:GZIP. It’s hard to argue with that approach.

Optimizing DEFLATE

No matter which DEFLATE-based Content-Encoding you choose, the quality of the implementation underlying DEFLATE implementation is key to reducing the size of the data stream. Many developers mistakenly assume that “DEFLATE is DEFLATE”, and any compressor that implements the algorithm will get the same result. This is a mistake.

Instead, think of DEFLATE’ing as solving a maze. A really really complicated maze. You might run around as fast as you can and eventually stumble upon an exit, but that’s very different than finding the optimal path. If you instead spent much longer, frequently retracing your steps to find out whether there’s a shortcut you missed, you may find a much shorter path. DEFLATE works much the same way—you can expend more resources (CPU time and memory) in the compression process finding the optimal compression choices. The best part is that expending resources to optimize DEFLATE compression doesn’t typically increase decompression time—uncompressing DEFLATEd content is a comparatively straightforward process.

For static resources that will be reused often (think of jQuery.js and other frameworks) you should be delighted to trade a one-time compression cost for a millions-of-times transfer size savings. Fortunately, folks at Google have done the hard work of making a great DEFLATE implementation, called Zopfli; you can read more about it and its real-world savings over here. Unfortunately, far too few teams have integrated Zopfli into their workflow; even Google hasn’t gotten around to using it for most of their resources… yet.

If Microsoft were to use Zopfli when building the browser-based versions of Office, they’d see significant savings; three of their largest files shrink by just over 4% each:

File Original Size Served (gzip) Size Zopfli Size Savings
WordViewer.js 661,771 171,189 164,272 4%
Ewa.js 729,547 202,400 193,822 4.2%
WordEditor.js 1,876,196 482,125 462,147 4.1%

Of course, you won’t always be able to use Zopfli to compress your resources—the compression tradeoff isn’t appropriate for dynamically-generated responses which will only be served once. But you can use it for more than just your HTML, CSS, and JS. Read on for more details.

Exotic Encodings

One of the first criticisms of Zopfli was “it seems like an awful lot of effort for a small improvement. Perhaps it’s time to add a better compression method.” And this criticism is as valid to a certain extent.

bzip2

For instance, the first versions of Google Chrome added support for bzip2 as a HTTP Content-Encoding, because (quoting a Google engineer) “well, we had the code laying around.” Bzip2 yields much better compression than even Zopfli-optimized DEFLATE:

File Zopfli Size BZIP2 Size Savings
WordViewer.js 164,272 139,393 15.1%
Ewa.js 193,822 163,204 15.8%
WordEditor.js 462,147 396,897 14.1%

And bzip2 doesn’t even offer the highest compression ratios: lzma2 showed even better results when we looked at it back in the IE9 timeframe.

The challenge with more advanced compression schemes is that there are billions of clients that support DEFLATE-based schemes, and as we’ve seen previously, most implementations haven’t yet optimized their use of DEFLATE. Making matters worse are intermediaries: Google reportedly encountered network proxies and security applications that would corrupt bzip2-based traffic.

Xpress

Some Microsoft products (Exchange, Software Update Services, etc) use Content-Encoding: xpress, a scheme based on LZ77 (like DEFLATE), optimized for compression speed. Xpress encoding is not used by Internet Explorer, WinINET, WinHTTP, or System.NET, the dominant HTTP implementations on Windows. When run on Windows 8 or later, Fiddler can decompress Xpress-encoded content using the native libraries.

SCDH

Back in 2008, Google proposed the SCDH (Shared Dictionary Compression over HTTP) Content-Encoding, and support has subsequently been added to Chrome and Android. Despite offering large potential savings, this lossless compression scheme is not broadly used and is comparatively complex to implement. It also has privacy implications.

Compress

The only non-DEFLATE-based HTTP-specification defined encoding (“Content-Encoding: compress”) is based on the LZW compression algorithm. Compress is not broadly supported (it doesn’t work in any major browser) and a quick test suggests that it’s not even as effective as a basic DEFLATE encoder.

What Gets Compressed

In general, HTTP Content-Encoding based compression applies to only the response body. This means that HTTP headers are not compressed, a significant shortcoming in the compression scheme. For instance, I recently observed Facebook delivering a 49 byte GIF file with 1050 bytes of HTTP headers, an overhead of over 2000% for a single file. Similarly, Content-Encoding is rarely applied to request bodies.

TLS Compression

When negotiating a HTTPS connection, the client may indicate support for automatic compression of all data on the connection, which has the benefit of applying to both headers and bodies of both the request and the response. TLS-based compression was not broadly implemented (Microsoft products have never supported it) and in 2012 it was disabled entirely by major products to address an exploit called CRIME (Compression Ratio Info-leak Made Easy).

HTTP/2 Compression

The HTTP/2 draft 14 enables header compression using an algorithm called HPACK, designed to combat the CRIME exploit against the SPDY protocol (which used DEFLATE to compress header fields). Draft 12 of the specification removed per-frame GZIP compression for data; you’ll still use Content-Encoding.

Compressing Request Bodies

In theory, HTTP allows clients to compress request bodies using the same Content-Encoding mechanism used for HTTP responses. In practice, however, this feature is not used by browsers and is only rarely used by other types of HTTP clients.

One problem is that a client does not know, a priori, whether a server accepts compressed requests. In contrast, a server knows whether a client accepts compressed responses by examining the Accept-Encoding request header. Many servers do not accept compressed requests, although some have no objection (e.g. onedrive.live.com allows them). One consequence of this is that HTML forms do not (yet?) expose any way for authors to specify that compression of the request body should be undertaken.

Modern web applications can workaround the shortcomings in browser APIs with script:

  1. Use HTML5 File API to load a file for upload into JavaScript
  2. Compress it using a script-based compression engine (of many, Pako and compressjs look nice)
  3. Upload the compressed array to the server.

You can use Fiddler to see whether your server accepts request bodies with Content-Encoding; click Rules > Customize Rules. Scroll to OnBeforeRequest and add the following code inside the function:   

  if (oSession.HTTPMethodIs(“POST”))
   {
     oSession["ui-backcolor"] = “yellow”;
     oSession.utilGZIPRequest();  
   }

If the server accepts the upload, you should see a normal HTTP/200 response (although you may wish to ensure that the server properly recognized the compression and didn’t just treat it like binary garbage). If the server doesn’t accept compressed uploads, you will likely see a HTTP/400 or HTTP/500 error code.

Compression Bombs

One reason that servers might be reluctant to support compressed uploads is the fear of “Compression bombs”. The DEFLATE algorithm allows a peak compression ratio approaching 1032 to 1, so a one megabyte upload can explode to 1 gigabyte. Such attacks are generally possible against client applications like browsers, but tend to be much more potent against servers, where a single CPU is required to serve thousands of users simultaneously.

Protecting against maliciously crafted compression streams requires additional diligence on the part of implementers.

Best Practice: Minify, then Compress

Because DEFLATE yields such great reductions in size, you might be thinking “I don’t need to minify my assets. I can just serve them compressed.”

From a networking point of view, you’re mostly correct: the bytes-on-wire for DEFLATE will likely be nearly the same as DEFLATE alone. However, there are other concerns:

Memory: The browser must decompress your stylesheets, script, and HTML to strings, so a 50kb script file, compressed to 5kb, still uses 50kb of memory after it has been decompressed. If you can minify the script to 40kb before compressing, you can save at least 10kb of memory on the client (and likely more). This is particularly important on memory-constrained mobile devices.

Disk Cache: Some browsers store compressed responses in the cache in compressed form (and Firefox will even proactively compress uncompressed responses). However, Internet Explorer removes all Content-Encoding when writing to the cache file, meaning that your 50kb script file will occupy 50kb of the client’s cache. Smaller files will reload faster and are less enticing candidates for eviction when cache space is reclaimed.

File Formats

As mentioned previously, many file formats internally use DEFLATE compression; a great example is the PNG image format. Unfortunately, many generators of those formats opt for minimum compression time, failing to achieve maximum compression.

The next release of Fiddler will include PNG Distill, a simple tool that removes unnecessary metadata chunks and recompresses image data streams using Zopfli. Zopfli-based recompression often shaves 10% or more from the size of PNG files.

image

The “Minify-then-compress” Best Practice applies to image file types as well. While large fields of empty pixels compress really well, the browser must decompress those fields back into memory.

Beyond PNG, the WOFF font format also uses DEFLATE. Google recently shrank their WOFF font library by an average of 6% by reencoding their internal DEFLATE streams using Zopfli. Not content to stop there, Google has also introduced the WOFF2 format, which utilizes a new font-specific algorithm known as Brotli, yielding size reductions around 20% (Currently usable in Chrome, Opera, and Firefox dev builds; see CanIUse.com).

Compression-Related Trivia

Here are a few points of compression-related arcana that have come up over the years…

1. The HTTP decompression implementation used by Internet Explorer 6 and earlier was inside URLMon; the WinINET HTTP stack did not itself support decompression. As a consequence, there was a “dance” whereby WinINET would download the compressed version and ask URLMon to decompress the response in place, replacing the original compressed body. This “dance” was buggy in many corner cases, and led to problems where content could not be decompressed, or Vary headers were ignored, or similar.

In Internet Explorer 7, decompression was moved from URLMon down into WinINET, eliminating the “dance” and slaying many bugs.

2. Unfortunately, the fact that decompression happens so low in the stack causes its own minor problems, like the F12 Developer Tools not being aware of the use of Content-Encoding.

3. Some Apache servers will mistakenly serve .tar.gz files with a Content-Encoding: gzip header, leading the browser to eagerly decompress the downloaded file into a raw tar file with now-misleading filename extension. Firefox works around this issue by ignoring the Content-Encoding header if the response Content-Type is application/x-gzip. Internet Explorer 11 recently and briefly had a regression in its handling of related scenarios.

 

-Eric
MVP, Internet Explorer

BizTalk Server is not supported with SQL Always on

 

At the time of writing this post  (October 2014), BizTalk Server is not supported with the feature SQL Server Always ON Triste enabled.

SQL Server Always ON

The AlwaysOn Availability Groups feature is a high-availability and disaster-recovery solution that provides an enterprise-level alternative to database mirroring. Introduced in SQL Server 2012, AlwaysOn Availability Groups maximizes the availability of a set of user databases for an enterprise. An availability group supports a failover environment for a discrete set of user databases, known as availability databases, that fail over together. An availability group supports a set of read-write primary databases and one to eight sets of corresponding secondary databases. Optionally, secondary databases can be made available for read-only access and/or some backup operations.

An availability group fails over at the level of an availability replica. Failovers are not caused by database issues such as a database becoming suspect due to a loss of a data file, deletion of a database, or corruption of a transaction log.

BizTalk, DTC and Always ON

When BizTalk Server and SQL Server are installed on separate computers, Distributed Transaction Coordinator (MS DTC) handles the transactions between the computers. As a result, the SQL Server AlwaysOn feature is not supported. The SQL Server AlwaysOn feature does not support MSDTC transactions.

This is because transaction atomicity/integrity cannot be guaranteed for distributed transactions: After a failover, the new principal server/primary replica is unable to connect to the distributed transaction coordinator on the previous principal server/primary replica. Therefore, the new principal server/primary replica cannot obtain the transaction status. In other words, after failover, the new principal, contacts MS DTC but MS DTC has no knowledge of the new principal server, and it terminates any transactions that are “preparing to commit,” which are considered committed in other databases.

This might cause many database inconsistences across all BizTalk Databases.

To understand how BizTalk and MSDTC work together see a post I wrote time ago here –> http://blogs.technet.com/b/biztalkpfe/archive/2011/01/12/understanding-msdtc-amp-biztalk.aspx#comments

 

References:

AlwaysOn Availability Groups (SQL Server) http://msdn.microsoft.com/en-us/library/hh510230.aspx

Understanding MSDTC & BizTalk http://blogs.technet.com/b/biztalkpfe/archive/2011/01/12/understanding-msdtc-amp-biztalk.aspx#comments

What’s New in BizTalk Server 2013 and 2013 R2  http://msdn.microsoft.com/en-us/library/jj248703(v=bts.80).aspx

D ??????????????

このポストは、10 月 6 日に投稿された D-Series Performance Expectations の翻訳です。
新しい D シリーズの VM は、高速なローカル (一時) ストレージまたは CPU を必要とするアプリケーションで優れたパフォーマンスを実現します。ただし、最適なパフォーマンスを得るためには、システムが構成されるしくみについて多少理解することが重要です。
クラウドは共有環境であるため、VM の大半は 1 つ以上の他の VM とサーバーを共有しています。1 つの VM が共有リソースを妥当な分量以上に使用することを防ぐため、VM が利用できる仮想 CPU のコア数、RAM、ローカル ディスクの容量、IO スループットには VM の種類に応じて制限が適用されています。これらの制限により、すべてのユーザーがシステムから一貫性の高い予測可能なパフォーマンスを得ることができます。
新しい D シリーズには、VM の種類に応じてローカル SSD に以下のようなコア数と RAM が割り当てられ、容量およびパフォーマンスの制限が適用されています。

…(read more)

CppCon Videos Available

In case you missed CppCon or attended but missed a few sessions (or want to review), videos for most of the talks are available on You Tube . There is plenty there for developers of all experience and temperament including:

C++ in Microsoft Office: How Microsoft Uses C++ to Deliver Office Applications Across Windows, Apple, Android, and Web Platforms ( part II ), Zaika Antoun

From the Dropbox Trenches: A Deep Dive into Two Cross-Platform Mobile Apps Written in C++ , Steven Kabbes…(read more)

Facebook Syndication Error

This feed URL is no longer valid. Visit this page to find the new URL, if you have access: <a href=”https://www.facebook.com/profile.php?id=306705402722523″>https://www.facebook.com/profile.php?id=306705402722523</a> …read more…(read more)