Where am I?

XQuery 3.0 introduces new syntax for "group by". At this time, MarkLogic Server 4.2 is the latest release, and it doesn't have support for that syntax. So how can we implement "group by" when writing XQuery for MarkLogic?

Let's start with the W3C use cases. First, let's fetch the sample data and put it into MarkLogic. We can do that using cq. I'll leave out the schemas, since we don't need those. I also won't be exhaustive about optimizing every expression in these examples: suffice to say that there is room for even more improvement.

Sorry about the long block of code, but we need those documents. Paste that into cq, evaluate it, and you should get the empty sequence. That means your documents were inserted correctly: you can use the 'explore' link to check.

Now we can write some queries. Here is the first use case (Q1).

And the result should look like this:

We can't write XQuery 3.0 using XQuery 1.0 — but we can get the same result using an extra distinct-values step.

This code is a little awkward, though. Instead of looping through the records once, we have to perform a database lookup on each product name. Normally this would be an unavoidable cost, and perhaps a reason to look forward to XQuery 3.0. But MarkLogic gives us a way to cheat, and use an accumulator model to get the same result more quickly. I'm talking about maps.

This produces the same output, and will scale better the distinct-values() version would. Of course it is also less portable. But database application developers often have to implement non-portable optimizations, and the less portable code can be segregated into its own library modules.

Now let's look at the next example (Q2).

Expected result:

Here is a solution using maps:

Again, this solution produces the same results. This time we had two elements in the grouping key, and the map key must be a string. So we had to use an old database trick and concatenate the two values with a known delimiter. Naturally we have to be careful in our choice of delimiter.

For the remaining queries, I'll skip the W3C examples and output XML. Here are my solutions. Again, these return the desired results, but could benefit from more optimization work.

This final use-case is kind of odd, because the sample code works if you simply comment out the "group by". In other words, the sample data only contains one group. But I reimplemented it anyway.

That's it. I hope this was worth your time.

When AWS EC2 tells you that "requested instance type (cc1.4xlarge) is not supported in your requested Availability Zone (us-east-1b)", what they really mean is that you are using the wrong AMI. Switch to an HVM AMI. This enables support within Xen for Hardware Virtual Machine (HVM), using AMD SVM or Intel VT-x instructions.

Tomorrow's event prodded me into setting up IPv6 at home, where I use openwrt. The tutorial I found was helpful: I just had to change the interface names. On my system eth0.1 was eth1, and 6rdtun was called 6to4. Comcast's test page says I'm up and working. I can see the unicorn too.

Visit to ipv6-test.net for more tests.

Intel's latest SSD pricing isn't as much of a shift as I had hoped for. As I see it, they have gone from $2.75/GB to $2.50/GB for enterprise-class SSD devices, and capacities have grown to 230-GB per device. That's an improvement, and the performance looks good. The combination of SATA-3 and lower failure rates than consumer-grade SSD may also help justify the price.

While SSD is now the logical choice over 15k-rpm disks, very few deployments use those. Instead, 10k-rpm are the workhorse for disk-heavy enterprise applications. SSD is getting closer, but still costs at least twice as much. Brand-name 10k-rpm SATA disks are available for $0.72-$1 per GB, with capacities up to 600-GB per spindle. It will take a lot of performance-related pain to cross that gap.

Part 5 in a series by Mel Gorman describes how to measure the potential benefit from hugepages. The results match up reasonably well with CPU-intensive synthetic benchmarks on linux, which tend to show 10-15% improvement over ordinary pages.

The larger impact may be to application environments under heavy memory pressure. The OS can swap everything else out, but hugepage allocations are pinned. This is a double-edged sword. Preventing swapping may benefit some environments (cf vm.swappiness). But imagine a situation where you have 8-GB RAM and designate 4-GB for huge pages, but only use 2-GB. Now the OS has only 4-GB to manage, and the free 2-GB in huge pages are effectively wasted. If the system comes under memory pressure, that could lead to swapping or activate the OOM killer.