Async Recursion with backoff

It’s been a while since I published anything. More than three years! A lot of things happened since then. The most relevant to mention in the beginning of this post is that I have been super busy building a lot of cool tech with a very talented team here at EPAM Anywhere. We are doing full-stack Typescript with next.js and native AWS serverless services and can’t get enough of it. This experience has been challenging me to learn new things every day and I have a lot to share!

When you work with AWS, you will certainly use aws-sdk and APIs of different services. Need to send a message to an SQS queue? That’s an HTTP API call and you will use sdk. Need to update a document in DynamoDB? The same. Need to push a message to the Firehose? The same. Many of these APIs have their batch equivalents:

These batch APIs will throw if something fundamental is wrong. Say your auth is not good or you don’t have enough permissions or you don’t have the connectivity to the service. If the sdk connected successfully to the service but failed to perform some or all of the operations in your batch, the operation won’t throw. It will return an object that tells you which operations succeeded and which ones failed. The most likely reason to get partial failures is due to throttling. All of these APIs have soft and hard limits and sooner or later you will attempt to do more than AWS feels comfortable letting you get away with.

We learned it the hard way. It’s all documented, obviously, but things like this one are only obvious in hindsight. Let me show you a neat technique to batch safely:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
interface DynamoBatchDeliveryOptions {
readonly db: DocumentClient;
readonly table: string;
readonly retries?: number;
readonly backoff?: BackoffStrategy;
}

/**
* Perform batch write/delete to the DynamoDB. All records that failed will be retried up to five times with a backoff.
* The method will throw if we failed to deliver the batch after the specified number of retries. Default is 5
*
* This factory method returns a reusable function that you can use over and over again if you are sending batches to the same `table`.
*
* **NOTE** We are not checking any limits imposed by AWS/Dynamo. As of the time of this writing:
* - 25 requests per batch
* - max document size is 400Kb including attribute name lengths
* - the total size of items written cannot exceed 16Mb (400x25 is less btw)
*
* @param db instantiated document client
* @param table the name of the DynamoDB table to work with
* @param retries how many times to retry. default is 5
* @param backoff backoff strategy that can be calculated based on the attempt number. the default is 100ms * attempt
* @returns the reusable function that you call with your batch details
*/
export const deliverBatchToDynamo =
({ db, table, retries = 5, backoff = attemptTimesOneHundredMs }: DynamoBatchDeliveryOptions) =>
(batch: DocumentClient.WriteRequests): Promise<void> => {
const run = async (batch: DocumentClient.WriteRequests, attempt: number): Promise<void> => {
if (attempt > retries) {
throw new Error(`Failed to deliver batch after ${attempt} attempts`);
}

const { UnprocessedItems } = await db.batchWrite({ RequestItems: { [table]: batch } }).promise();

if (UnprocessedItems) {
const retry = UnprocessedItems[table];
if (retry?.length) {
return new Promise((resolve) => setTimeout(() => resolve(run(retry, attempt + 1)), backoff(attempt)));
}
}
};

return run(batch, 1);
};

If you want to learn more about the technique, please checkout the full version of this post on our engineering blog: https://anywhere.epam.com/en/blog/async-recursion-with-backoff.

Till next time!

Understanding Date Ranges in Your Chatbot

When your chatbot performs tasks of a personal assistant like scheduling meetings or generating reports, you need to make sure it can understand dates and date ranges.

Step 1. Resolve

LUIS has a set of pre-built entities to recognize date and time (builtin.datetime). It will understand when your users say tomorrow, October 1st or next week, for example, and will convert that to a date or a duration. Couple examples:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// tomorrow
"resolution": { "date": "2016-11-20" }

// last quarter
"resolution": { "date": "XXXX-Q4" }

// last year
"resolution": { "date": "2015" }

// last two years
"resolution": { "duration": "P2Y" }

// last week
"resolution": { "date": "2016-W45" }

// past three weeks
"resolution": { "duration": "P3W" }

// this month
"resolution": { "date": "2016-11" }

// last ten months
"resolution": { "duration": "P10M" }

Unfortunately, the only quarter-based duration LUIS understands right now is last quarter. It doesn’t recognize this quarter, next quarter, or plurals like last three quarters.

As you can see, the resolutions are indicative, use different formats, and need to be parsed to get converted to dates and date ranges.

Step 2. Parse

When LUIS detects a datetime entity (e.g. tomorrow) it will send back the resolution along with the extracted entity itself (the word tomorrow in this case).

First, I try to understand what time span the user asked about:

1
2
const span = 
['day', 'week', 'month', 'quarter', 'year'].find(s => entity.match(s));

Then I parse the dates and durations with moment:

1
2
3
4
5
6
7
8
9
10
11
12
13
const moment = require('moment');

// date
const resolved = resolution.date.replace('XXXX', moment().year());
const date = moment(resolved, ['YYYY-MM-DD', 'YYYY-Q', 'YYYY-W', 'YYYY']);

// duration
const duration = moment.duration(resolution.duration);
const sign = ['last', 'past', 'previous'].some(p => entity.match(p)) ? -1 : +1;
const date = moment().add(sign * duration.as('hours'), 'hours');

// normalized result
return date.startOf(span || 'day');

Step 3. Understand

Now we have the date representing the beginning of the period the user asked about. If today was Friday 11/18, for example, and you asked for last three weeks, the date would be Sun, Oct 23 (weeks start on Sunday in US unless you use isoweek with moment).

One date is not enough though for utterances like:

1
please generate a service cost report for the last two weeks

Your report generation service/API is likely to require a date range.

LUIS can also understand numbers spelled as digits like 2 or 5 or spelled as words like two or five. A phrase like last two weeks will produce two entities:

1
2
3
4
5
6
7
8
9
10
11
12
13
"entities": [
{
"entity": "two",
"type": "builtin.number"
},
{
"entity": "last two weeks",
"type": "builtin.datetime.duration",
"resolution": {
"duration": "P2W"
}
}
]

Last thing I need to do to understand the range, is to extract the number and do the date math:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
const moment = require('moment');
const builder = require('botbuilder');

const numbers = {
'one': 1,
'two': 2,
'three': 3,
// you got the idea
};

// the entity here is the 'builtin.number'
const range = builder.EntityRecognizer.parseNumber(entity)
|| numbers[entity]
|| 1;

const end = moment(date)
.add(range, span)
.subtract(1, 'day')
.endOf('day');

And that’s it. Now last three weeks is understood as 10/23 - 11/12. And last quarter will be 10/1 0:00 - 12/31 23:59.

Intent Recognizers For Your Chatbot

Two weeks ago I attended API Strat in Boston where I gave a talk on cognitive APIs and conversational interfaces and showed and explained an e-commerce chatbot that I built. My presentation is on slideshare. I have learned a lot about chatbots and now I feel an urge to write about it.

Skype conversation excerpt

Intents

My bot is using the intent dialog from the Microsoft Bot Framework:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
const bot = new builder.UniversalBot(...);
const intents = new builder.IntentDialog(...);

intents.matches('Greeting', '/welcome');
intents.matches('ShowTopCategories', '/categories');
intents.matches('Explore', '/explore');
intents.matches('ShowProduct', '/showProduct');
intents.matches('AddToCart', '/addToCart');
intents.matches('ShowCart', '/showCart');
intents.matches('Checkout', '/checkout');
intents.matches('Reset', '/reset');
intents.matches('Smile', '/smileBack');
intents.onDefault('/confused'); // no intent recognized

bot.dialog('/', intents);

bot.dialog('/confused', [
function () {
session.endDialog('Sorry, I didnt understand you');
}
]);

The intent dialog associates a user’s intent like Explore or Checkout with a specific dialog that knows how to respond.

It feels very much like routing in a web framework where given a specific URL pattern, the request will be routed to a controller that knows how to handle it.

Users don’t spell out their intents like that though. And so the first thing my bot needs to do is to learn to recognize them. The simplest way to trigger a dialog handler in response to a users utterance is by matching it with a regex. A more sophisticated logic requires an intent recognizer.

Intent Recognizers

An intent recognizer is basically a service that can understand users’ utterances. Given a text message it will return a list of intents that it inferred from it along with supporting entities. Here’s how it looks in LUIS (language understanding service from Microsoft):

LUIS intents and entities

The Explore intent was recognized along with two supporting entities that I trained it for. Here’s another way of looking at it:

1
2
3
4
5
curl -v "https://api.projectoxford.ai/luis/v2.0/apps/{app-id}" 
-H "Content-Type: application/json"
-H "Ocp-Apim-Subscription-Key: {subscription-key}"
-G
-d "q=I am looking for touring bikes. Do you have some?"

And the response:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{
"query": "I am looking for touring bikes. Do you have some?",
"topScoringIntent": {
"intent": "Explore",
"score": 0.9994699,
"actions": [
// ...
]
},
"entities": [
{
"entity": "touring",
"type": "Detail",
"score": 0.9710912,
// ...
},
{
"entity": "bikes",
"type": "Entity",
"score": 0.943606555,
// ...
}
],
// ...
}

Microsoft Bot Framework comes with built-in support for LUIS in the form of LuisRecognizer

Custom Recognizers

Not every thing your users say has to be sent to a natural language service to extract the intent. Buttons and tappable images can post back bot-specific commands like /show:123456789, for example, that you can easily recognize with a regex. Also, if you want your bot to smile back at a smile sent to it, you don’t need to train a linguistic model either.

It turns out, building your own recognizer is not hard at all. I have built a few for my e-commerce bot and here’s how it works.

First, know that the Bot Framework supports sending a message through a number of recognizers at the same time. You can chain them or run them all in parallel:

1
2
3
4
5
6
7
8
9
10
const intents = new builder.IntentDialog({
recognizers: [
commands,
greeting,
smiles,
new builder.LuisRecognizer(process.env.LUIS_ENDPOINT)
],
intentThreshold: 0.2,
recognizeOrder: builder.RecognizeOrder.series
});

The recognizer itself is a very simple interface with only one method - recognize. Here’s how you would detect a smile, for example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
module.exports = {
recognize: function(context, callback) {
const text = context.message.text;
const smiles = text.match(/<ss type="(\w+?)">(.+?)<\/ss>/);

if (smiles) {
callback.call(null, null, {
intent: 'Smile',
score: 1,
entities: [
// smiles[1] and smiles[2]
// have the details you need to smile back
]
});
} else {
callback.call(null, null, {
intent: null,
score: 0
});
}
}
};

And here’s another one that understands commands:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

const commands = {
parse: function (context, text) {
const parts = text.split(':');
const command = parts[0];

const action = this[command] || this[command.slice(1)];
if (!action) {
return unrecognized;
} else {
return action.call(this, context, ...parts.slice(1));
}
},
// ...
}

module.exports = {
recognize: function (context, callback) {
const text = context.message.text;

if (!text.startsWith('/')) {
callback.call(null, null, unrecognized);
} else {
callback.call(null, null, commands.parse(context, text));
}
}
};

That’s it for now but there is more to come. Stay tuned!

How To Promisify Moltin APIs

If you’ve read my last post, then you know that I am having all kinds of geeky fun with Moltin and its APIs. Today I will show you how you can quickly promisify them all.

Moltin APIs

Moltin APIs are all asynchronous HTTP calls with very lightweight wrappers for JavaScript, Python, and many other languages. I am using it with node.js and the main pattern is fairly straightforward (look for js examples).

I am writing a batch import to create a playground product catalog so I find myself doing a lot of this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
const inventory = Promise.all(goGetTheData()); 

inventory.then((data) => {
return Promise.all(data.products.map(product => {
return new Promise((resolve, reject) => {
moltin.Authenticate(function() {
moltin.Product.Create({
// .. product attributes
}, (result) => {
resolve(result);
}, (error, details) => {
reject(details);
});
});
});
}));
}).then((products) => {
return Promise.all(products.map(p => {
return new Promise((resolve, reject) => {
moltin.Authenticate(function() {
// ...
});
});
}));
}).then((modifiers) => {
// ... you got the idea, a lot of noise
});

I wish I could instead write:

1
2
3
4
5
6
7
inventory.then((data) => Promise.all(data.products.map(p => {
return moltin.Product.Create(...);
}))).then((products) => Promise.all(products.map(p => {
return moltin.Modifier.Create(...);
}))).then((modifiers) => {
// ... a lot cleaner and more readable, isn't it?
});

Promisification

There’s moltin-util on NPM that uses Promises but it seems to introduce a new API and I would like to retain the original. Here’s what I quickly put together and I now wonder if it’s worth posting to NPM. Is it? Let me know!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
const request = require('request');
const fs = require('fs');

const promisify = (moltin) => {
const promisified = {}

const executor = (actor, action) => function () {
const args = [...arguments];

let success = (result, pagination) => {
if (result && pagination) {
result.pagination = pagination;
}

return result;
};

let error = (error, details) => details;

if (typeof (args[args.length - 1]) === 'function') {
if (typeof (args[args.length - 2]) === 'function') {
error = args.pop();
success = args.pop();
} else {
success = args.pop();
}
}

return new Promise((resolve, reject) => {
moltin.Authenticate(function () {
actor[action].call(actor, ...args,
(result, pagination) => {
resolve(success.call(null, result, pagination));
},
(err, details) => {
console.error(details);
reject(error.call(null, details));
});
});
});
};

Object.keys(moltin)
.filter(key => key !== 'options' && typeof (moltin[key]) === 'object')
.forEach(member => {
promisified[member] = {};
let actor = moltin[member];

Object.keys(actor.__proto__)
.concat(Object.keys(actor.__proto__.__proto__))
.filter(action => typeof (actor[action]) === 'function')
.forEach(action => {
promisified[member][action] = executor(actor, action);
});
});

return promisified;
}

module.exports = function (moltin) {
return promisify(moltin);
};

My code is now a whole lot cleaner and smaller too. I will soon post it on Github so stay tuned! Here is, for example, how I would go about deleting a whole bunch of products:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
const moltin = require('moltin')({
publicId: process.env.MOLTIN_PUBLIC_ID,
secretKey: process.env.MOLTIN_SECRET_KEY
});
const moltin_p = require('./promisify-moltin')(moltin);

moltin_p.Product.List(null)
.then((products) => Promise.all(products.map(p => {
console.log('Requesting a delete of %s', p.title);
return moltin_p.Product.Delete(p.id);
})))
.then((result) => {
console.log('Deleted %s products', result.length);
})
.catch((error) => {
console.error(error);
});

Sequencing Asynchronous Calls in JavaScript

I am playing with Moltin for my upcoming talk on the API Strategy conference and it generates all kinds of blog posts ideas.

Context

Moltin is the API-first (or I would even argue the API-only) commerce platform. A “new kid on the block”, a recent Y Combinator graduate with a little more than $2M in seed funding. My talk is about cognitive APIs and smarter apps and I will be using a conversational e-commerce chatbot as an example. I picked Moltin as my commerce backend because it’s ridiculously easy to get started with, requires no upfront setup, and seems to provide a simple and yet a rich API that covers all my scenarios. Plus, their free tier gives me 30,000 requests per month.

You can’t transact with a commerce platform if it doesn’t have a product catalog. Products have variants (e.g. a t-shirt can come in different sizes and different colors) and different commerce platforms approach setting up this hierarchy differently. In Moltin, you first create a main product. Then you add modifiers (in my case - color and size). And then you add variations for each modifier. Moltin will then create the actual variants matrix behind the scenes. If, for example, you add blue, red, and white variations to the color modifier and S, M, L to the size modifier, you will end up with a nine total variations (every size available in every color).

Moltin APIs

Moltin APIs are lean HTTP endpoints that understand application/x-www-form-urlencoded and multipart/form-data and return back JSON. The team also supplies lightweight wrappers for JavaScript, Python, and other languages. Here’s, for example, how the creation of a variation in JavaScript looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
const moltin = require('moltin')({
publicId: process.env.MOLTIN_PUBLIC_ID,
secretKey: process.env.MOLTIN_SECRET_KEY
});

moltin.Authenticate(function() {
moltin.Variation.Create(productId, modifierId, {
title: value
},
function(result){
// result is the successfully created variation
},
function(error, details) {
// oops
});
});

I am scripting the creation of the product catalog using Adventure Works as my sample dataset so I need to run a lot of these asynchronous callback style request in order. I do it with Promises:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
const addVariation = (productId, modifierId, value) => {
return new Promise((resolve, reject) => {
moltin.Authenticate(function() {
moltin.Variation.Create(productId, modifierId, {
title: value
},
function(result){
resolve(result);
},
function(error, details) {
reject(details);
});
});
});
};

And now I can chain all my actions with .then().

Problem

I faced an interesting challenge as I was creating the variations for my products. Here’s how I do it. First, I figure out what modifiers I need to create and then for each modifier I collect the values. The result looks something like this:

1
2
3
4
5
6
7
const mods = [{
title: 'color',
values ['red', 'blue', 'white']
}, {
title: 'size',
values ['S', 'M', 'L']
}];

Now I can recursively .map() this structure into an array of Promises each creating a required variation in Moltin:

1
2
3
4
5
6
7
8
9
10
// somewhere on the chain
.then((mods) => {
return Promise.all(_.flatMap(mods, mod => {
return mod.values.map(value => {
return new Promise((resolve, reject) => {
// create the variation in Moltin
});
});
}));
})

“Looking good”, I thought, until I found out that creating all the variations asynchronously in no particular order and maybe even in parallel confuses the logic on the Moltin side that creates the product matrix (details).

Since I can’t trigger a matrix rebuild via the API, the solution was to sequence the variants creation.

Solution

Instead of just mapping the modifiers to a list of Promises running somewhat concurrently, I I needed to chain variants creation one after another. I also needed to collect all created variations into a list for the next step in the bigger chain.

Nested reduce to the rescue. First, the addVariation now keeps track of the results:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
const addVariation = (productId, modifierId, value, bag) => {
return new Promise((resolve, reject) => {
moltin.Authenticate(function() {
moltin.Variation.Create(productId, modifierId, {
title: value
},
function(result){
bag.push(result)
resolve(result);
}
// error handling skipped
});
});
};

And the Promie.all() has to convert into a linear chain of promises each creating a single variation:

1
2
3
4
5
6
7
8
9
10
11
12
// somewhere on the chain
.then(mods => {
var variations = [];

const chained = mods.reduce((chain, mod) => {
return mod.values.reduce((chain, value) => {
return chain.then(() => addVariant(productId, mod, value, variations));
}, chain);
}, Promise.resolve());

return chained.then(() => Promise.resolve(variations));
})

Works great but I feel like it can be cleaner with Rx. If you know how to convert this to observables and not explicitly manage the collection of created variants, please drop me a line. Thanks!