@guyro Using typica library I was able to get 12 SQS enqueues per second per thread.
This caught my interest, as in jclouds, we claim we have decent performance. Now's the time to put it to the test.
In order to prepare, I had some homework to do. Firstly, we hadn't yet implemented SQS. After about 3 hours, I created the new client (not 100% finished, but enough to start).
Next, I needed to setup a test case, which was sure to get top performance. For this, I completely turned off logging and switched to our asynchronous interface. Later in this blog, I'll show you how this test was coded. For now, let's run it!
Here's what you need to do:
- Setup an instance of ec2 running ubuntu to make it easy (ami-7e28ca17)
- Install java (apt-get install -y openjdk-6-jdk)
- Download the test jar (wget http://jclouds.googlecode.com/files/jclouds-speedtest-sqs.jar)
- Run the test (java -Xms128m -Xms128m -jar jclouds-speedtest-sqs.jar id secret queueName 250)
Here's an example of the output I received from an m1 small instance in the us-east region:
ubuntu@ip-10-242-197-159:~$ java -Xms128m -Xms128m -jar jclouds-speedtest-sqs.jar id key queueName 250
creating queue: queueName in region eu-west-1
creating queue: queueName in region us-east-1
creating queue: queueName in region us-west-1
COMPLETE: context: default, region: us-east-1, rate: 70.741370 messages/second
pausing 5 seconds before the next run
COMPLETE: context: default, region: eu-west-1, rate: 82.372323 messages/second
pausing 5 seconds before the next run
COMPLETE: context: default, region: us-west-1, rate: 104.777871 messages/second
pausing 5 seconds before the next run
deleted queue: queueName in region us-east-1
deleted queue: queueName in region us-west-1
Now, look at the performance from a c1 medium instance in the us-east region:
ubuntu@ip-10-244-243-82:~$ java -Djclouds.enterprise -Xms128m -Xms128m -jar jclouds-speedtest-sqs.jar id key testq 5000
creating queue: testq in region eu-west-1
creating queue: testq in region us-east-1
creating queue: testq in region us-west-1
COMPLETE: context: enterprise, region: us-west-1, rate: 487.092060 messages/second
pausing 5 seconds before the next run
COMPLETE: context: enterprise, region: us-east-1, rate: 720.357297 messages/second
pausing 5 seconds before the next run
COMPLETE: context: enterprise, region: eu-west-1, rate: 579.642940 messages/second
pausing 5 seconds before the next run
deleted queue: testq in region eu-west-1
deleted queue: testq in region us-east-1
deleted queue: testq in region us-west-1
Crazy difference, huh?! Unsatisfied, I turned up the volume to the highest. Here, I use the c1 xlarge instance and have to raise my file limit in preparation for tons of i/o.
root@domU-12-31-39-0C-A9-81:~# ulimit -n 10240
root@domU-12-31-39-0C-A9-81:~# java -Djclouds.enterprise -Xms256m -Xms256m -jar jclouds-speedtest-sqs.jar id key testq1 5000
creating queue: testq1 in region eu-west-1
creating queue: testq1 in region us-east-1
creating queue: testq1 in region us-west-1
COMPLETE: context: enterprise, region: eu-west-1, rate: 662.866234 messages/second
pausing 5 seconds before the next run
COMPLETE: context: enterprise, region: us-east-1, rate: 1594.387755 messages/second
pausing 5 seconds before the next run
COMPLETE: context: enterprise, region: us-west-1, rate: 1252.191335 messages/second
pausing 5 seconds before the next run
deleted queue: testq1 in region eu-west-1
deleted queue: testq1 in region us-east-1
deleted queue: testq1 in region us-west-1
The above performance is obtained with the standard java cached Executor and could probably be tuned to do much better.
I'd like to see results from different sizes and launched from different regions. Please let me know, if you get interesting results.
Dirty Details
Completely switching off logging isn't something I tend to recommend. However, if you want to see how we do it, here's what that code looks like:
context = SQSContextFactory
.createContext(System.getProperties(), accesskeyid, secretkey,
new NullLoggingModule());
Here, you'll see us configuring the SQS client, and also ensuring that we can override defaults using normal java system properties. The last bit overrides the normal logging system to essentially no-op.
Now, to make the test efficient, we need to use the asynchronous interface. This makes our tests more efficient, as it allows jclouds to manage threads in the best way possible. Here's how it looks to fire off a bunch of messages using the async interface:
// fire off all the messages for the test
Set responses = Sets.newHashSet();
for (int i = 0; i < messageCount; i++) {
responses.add(context.getAsyncApi().sendMessage(queue, message));
}
Here, we collect the responses from the firing. This will take less than a second to fire a few hundred messages. However, they are all in-progress and are now limited by processing, memory and network contention.
The next part we shuffle through and check the responses until they are all done, or we ran out of time.
do {
Set retries = Sets.newHashSet();
for (ListenableFuture response : responses) {
try {
response.get(100, TimeUnit.MILLISECONDS);
complete++;
} catch (ExecutionException e) {
System.err.println(e.getMessage());
errors++;
} catch (TimeoutException e) {
retries.add(response);
}
}
responses = Sets.newHashSet(retries);
} while (responses.size() > 0 &&
System.currentTimeMillis() < start + timeOut);
The rest of the code is normal stuff. Feel free to check out the real deal here.
I hope you enjoy this demonstration, and please do tweet me, if you have other ideas!
