Promiseland and Async/Await Kingdom

Last night I finally got a chance to publish the remaining setup scripts for my E-Commerce Chatbot. A few days ago, I added the script to load up products and variants into Azure Search and now also the catalog and historical transactions for Azure Recommendations. I basically had to script what I originally did as a one-off with curl.

Training the recommender model takes time and when you create a new recommendation build, it won’t be ready right away. I wanted my script to wait and keep polling the API until the training has finished. The whole script is basically a serious of asynchronous HTTP requests so I wired it all up as a chain of promises:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
sdk.model.list()
.then(({ models }) => {
// ...
}).then(() => {
return sdk.model.create(modelName, description);
}).then(() => {
return sdk.upload.catalog(...);
}).then(() => {
return sdk.upload.usage(...);
}).then(() => {
return sdk.build.fbt(...)
}).then(result => {

// <--
// ToDo: need to wait until the training is finished
// <--

}).then(() => {
console.log(`Set RECOMMENDATION_MODEL to ${model.id}`);
console.log(`Set RECOMMENDATION_BUILD to ${build.buildId}`);
}).catch(error => {
console.error(error);
});

You can see the full listing here.

Promiseland

Here’s how I implemented the wait-and-see:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// ...
}).then(build => {
const check = (timeout) => new Promise((resolve, reject) => {
setTimeout(() => sdk.build.get(model.id, build.buildId)
.then(response => {
if (!['NotStarted', 'Running'].includes(response.status)) {
console.log(`Build training finished: ${response.status}`);
resolve();
} else {
console.log(`Training is ${response.status}. Wait 30 seconds...`);
resolve(check(30000));
}
})
.catch(reject), timeout);
});

return check();
}).then(() => {
// ...
});

It’s basically a recursive promise. The function in the main then() will return a promise that will always resolve unless there’s an error, but the key is in what it will resolve with and how it runs. The function that the returned promise is wrapped around schedules itself via setTimeout() and exits the stack frame. Then, when the response is received, it will either resolve and signal that the training has complete, or it will resolve with another promise that will recursively repeat this process again. That another promise will basically insert itself into the main then chain and it will keep waiting until it resolves. Vicious circle.

It worked nicely and I even factored out the repeater so that my code looked like this:

1
2
3
4
5
6
7
8
9
10
11
// ...
}).then(build => {
return repeater.repeat(() => sdk.build.get(model.id, build.buildId), {
delay: 30000,
until: (response) => !['NotStarted', 'Running'].includes(response.status),
done: (response) => console.log(`Build training finished: ${response.status}`),
next: (response, delay) => console.log(`Training is ${response.status}. Wait ${delay / 1000} seconds...`)
});
}).then(() => {
// ...
});

Async/Await Kingdom

I really thought that I was very cleaver but then I decided to rewrite with async/await and run with the latest node that now natively supports it

Here’s what this code became:

1
2
3
4
5
6
7
8
9
10
11
12
let trained = false;
while (!trained) {
let check = await sdk.build.get(model.id, build.buildId);

if (!['NotStarted', 'Running'].includes(check.status)) {
trained = true;
console.log(`Build training finished: ${check.status}`);
} else {
console.log(`Training is ${check.status}. Wait 30 seconds...`);
await new Promise(resolve => setTimeout(resolve, 30000));
}
}

Here. Compare both versions: Original vs. Async/Await

It’s not even funny! The code is so boring now, boring and simple. Just like it should be. No need to be clever and I bet I will know exactly what it’s doing and why when I look at it a year later.

I have officially converted.

Cheers!

My Go-To Scaffold for React + API

A few weeks ago I found myself building a simple app, a prototype actually. It has a nice interface to request that a certain job (or multiple) be executed in the background. It also provides real-time updates about those jobs. Nothing that you can’t do with JavaScript. I quickly settled on a node.js back-end with a React front-end and a socket.io channel in between.

This post is about how I set up my solution and my dev environment to nicely bundle my client and my server together to make everything work smoothly locally (including the compound end-to-end debugging) as well as to be ready for production deployment to heroku.

The overall solution looks like this:

1
2
3
4
5
6
solution/
├── server/
│ ├── package.json
├── client/
│ ├── package.json
└── pacakge.json

The first three things that I did after I created the solution folder were:

1
2
3
& cd solution && npm init
$ create-react-app client
$ mkdir server && cd server && npm init

In development, I would like my client to start up using react-scripts with webpack server on :3000 with hot reloading and other awesomeness. In production, however, my server will be serving up all front-end assets. And it will run on a different port locally when executed side by side with the webpack server. From server/app.js:

1
2
3
4
5
6
7
8
9
10
11
12
const app = express();
app.use(express.static('./client/build'));
app.get('/', function (req, res) {
res.redirect('/index.html');
});

const http = require('http').Server(app);
const io = require('socket.io')(http, { path: '/api' });

http.listen(process.env.PORT || process.env.port || 3001, () => {
console.log('Express/Socket.io server is ready and is accepting connections');
});

First, I installed concurrently in the root of the solution so that I could run both server and client with one command:

1
$ npm install concurrently --save-dev

Then, I added the following command to the solution level package.json:

1
2
3
"scripts": {
"debug": "concurrently \"cd server && node --inspect=7244 app.js\" \"cd client && npm start\""
},

Now when I do npm run debug in the solution root, I get two processes spun up - one runs the server/app.js on :3001 and the other one runs the client on :3000. I also run server in debug mode and this will come handy when we get to setting up local debugging.

By the way, I used debug and not start command because I need npm start to be the way heroku launches this setup in production where server handles it all:

1
2
3
4
"scripts": {
"debug": "...",
"start": "node server/app.js"
}

I also need heroku to install all dependencies and build the front-end every time I push new version up. That’s one more npm command in the solution level package.json:

1
2
3
4
5
"scripts": {
"debug": "...",
"start": "...",
"postinstall": "cd server && npm install && cd ../client && npm install && npm run build"
}

The client expects socket.io to be accessible on the /api endpoint from the same server. From the App.js:

1
2
3
4
5
6
7
8
9
import io from 'socket.io-client';

class App extends Component {
constructor(props) {
super(props);

this.socket = io({ path: '/api' });
}
}

Easy in production setting where there is only one server. This is where proxy comes in to aid the development setup. We need to tell the webpack dev server to proxy everything it can’t handle to the server over at :3001. Need to add one little line to the client/pacakge.json:

1
2
3
{
"proxy": "http://localhost:3001/"
}

Last but not least, I would really like to be able to debug both client and server in one place. Visual Studio Code supports compound debugging since late last year. Here’s my launch configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
"version": "0.2.0",
"configurations": [
{
"name": "Server",
"type": "node",
"request": "attach",
"port": 7244
},
{
"name": "Chrome",
"type": "chrome",
"request": "launch",
"url": "http://localhost:3000",
"webRoot": "${workspaceRoot}/client/src"
}
],
"compounds": [
{
"name": "Hybrid",
"configurations": [
"Server",
"Chrome"
]
}
]
}

You will need the Debugger for Chrome extension. Now you can npm run debug and then F5 to attach to both processes.

Nirvana.


Anthony Accomazzo’s post - Using create-react-app with a server - made it very easy for me to set it all up. I am very happy to share it a little further with a thin layer of heroku and VS code debugging.

Enjoy!

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×