Introducing AirSim-js

ROS2JsGuy
7 min readJul 20, 2022

--

Recently I released AirSim-js, an open-source Nodejs client for AirSim (Aerial Informatics and Robotics Simulation). In this article I’ll introduce you to the cool AirSim environment and how to programmatically drive a car within AirSim using TypeScript and the AirSim-js package.

AirSimNH (neighborhood) Environment

Background

If you’re new to AirSim, it’s an open-source 3D simulation platform developed by Microsoft AI Research. Out of the box it includes high resolution models of flying drones and driving cars within worlds such as a simple blocks world, a busy city street environment, a residential neighborhood and even an abandoned amusement park. Within these environments, vehicles, weather, time of day and simulation rate of execution are configurable. Another cool feature of AirSim is it’s support for hardware-in-the-loop. But for developers, the coolest feature is that AirSim can be programmatically controlled through an RPC interface based on the msgpack-rpc protocol.

You can see AirSim in action in this short video.

Vehicle Sensors

AirSim vehicles can be outfitted with multiple types of sensors. The supported sensors are:

  • Barometer for altitude
  • Distance meter
  • GPS for position
  • IMU for acceleration and angular velocity
  • 3D LIDAR for point mapping
  • Magnetometer for heading

The drone default sensors are: barometer, GPS, IMU and magnetometer. The default car sensors are the lonely GPS sensor. You can easily create a new drone or car with additional sensors such as LIDAR.

Cameras and Images

In addition to sensors, vehicles are preconfigured with 5 cameras and named by their position, e.g., front_center, back_center, bottom_center or fpv. The specifics of each camera are customizable. Beyond the default cameras, new cameras can added to a vehicle or installed externally anywhere in the sim world. From any camera you can request images in the following formats: RGB, distance, infrared, optical flow or segmentation.

ROS Integration

AirSim includes an integration with the increasingly popular Robot Operating System (ROS). This is a useful capability being able to tie AirSim into the ROS ecosystem of navigation and control algorithms, e.g. SLAM.

Getting Started with AirSim

Coding with AirSim-js is simple but first you’ll need to install AirSim and become familiar with it.

Install and Launch the AirSim Blocks environment

The AirSim project provides prebuilt Linux and Windows binaries for several simulation worlds.

  1. Begin by downloading the Blocks world for your platform (Linux or Windows).

2. Unzip the Blocks.zip download to a folder of your choice.

3. From a terminal, navigate to the Blocks executable and launch it in Car view mode.

#Linux
> cd install_dir/Blocks/LinuxNoEditor/Blocks/Blocks.sh
#Windows
> cd install_dir\Blocks\WindowsNoEditor\Blocks\Binaries\Win64\Blocks.exe

On startup, AirSim will present the following view-mode selection dialog. Select Yes for Car view-mode.

The Blocks world window should appear containing a single car similar to this screenshot.

Blocks World in Car view-mode

4. Practice driving the car manually using the Arrow keys on your keyboard.

Coding with AirSim-js

Ping the Blocks environment

Our 1st coding task is to run an AirSim rpc `ping` command against our running Blocks process. For this you will need to have Nodejs installed on your system and basic knowledge of TypeScript. See the following directions if you need assistance with this prerequisite step.

Step-1: Node-TypeScript project setup

Create and initialize a Nodejs TypeScript package. Begin by entering the following commands from a terminal:

mkdir airsim-js-intro
cd airsim-js-intro
npm init -y
npm i airsim-js
npm i typescript @types/node --save-dev
npx tsc --init

Note — the Node-TypeScript project presented are for demonstration purposes only. Please don’t troll me.

Step-2: Coding ping.ts

Create a file named ping.ts and add the following TypeScript code to it. Review the comments for a narration of the code.

A special note about line-7. One of the responsibilities of the AirSim class is to serve as a factory for generic Vehicles, Cars and Multirotors. Therefore a class constructor of one of these vehicle classes is a required parameter. If you provide the constructor of Vehicle subclass such as Car or Multirotor it must match the view-mode of the AirSim world you are connecting. That is don’t run AirSim in Car mode and attempt to use the AirSim-js Multirotor API. Lastly, in the example, the code assumes that ping.ts be will executed on the same machine that is hosting the AirSim world. This is not a hard requriement as the AirSim constructor included parameters for a custom IP and port number.

For more AirSim class details see AirSim TSDoc.

/** 
* Create an AirSim client.
* AirSim serves as a factory for Vehicles, (e.g., Car, Multirotor)
* hosted in an AirSim environment. When constructing a new AirSim
* client instance you must provide the class of Vehicle hosted by AirSim.
* @param vehicleClass — The type of vehicle hosted by the AirSim server
* @param port — The AirSim server port number, default = 41451
* @param ip — The AirSim server IP address, default = 127.0.0.1
*/
constructor(
private vehicleClass: Constructor<T>,
public port: string | number = 41451,
public readonly ip = '127.0.0.1'
)

Step-3: Transpile ping.ts

This step converts the ping.ts file to the runnable ping.js javascript file.

npx tsc

Step-4: Run ping

Run the ping.js file using the Nodejs runtime.

node ping.js

If everything works correctly the console output should be similar to this:

> node ping.js
connected: true
Calling ping
result: true
Connected!
Client Ver: 1 (Min Req: 1)
Server Ver: 1 (Min Req: 1)

Drive the Car in the Blocks environment

Next let’s write some code to do something interesting like drive the car. Feel free to skip ahead to the code for driving the Blocks car.

The idea is to command the car’s steering angle and throttle such that the car turns in a circle. We will do that using a CarControls object. Here’s the definition of CarControls:

/**
* Car control settings
*/
export type CarControls = {
throttle: number, // 0.0–1.0
steering: number, // -1.0 to 1.0, left to right
brake: number, // 0.0–1.0
handbrake: boolean, // true = apply handbrake */
is_manual_gear: boolean, // true implies use manual_gear prop
manual_gear: number, // -1 to 5, 0 = neutral
gear_immediate: boolean // shift to manual gear setting now
}

We can create a CarControls instance from scratch or request the current CarControls from a car instance. Then we modify the subset of properties that represent our intentions as shown.

const controls = await car.getControls();
controls.throttle = 0.25; // forward at 1/4 speed
controls.steering = 1.0; // full right turn

Getting a handle to the Car

Before we can send commands to a car we need a handle to it. We can do this in 2 ways. The first is to access the car by name, assuming we know the car’s unique name. The second approach shown below is to request all the vehicles and find the target car in the list.

const cars = await airsim.getVehicles();
const car = cars[0]; // use the 1st car

API Control — Enable before use; Disable when not in use

Before we can send commands to a vehicle we need to enable it’s API. Then upon completion of our use of the vehicle’s api, we should always disable the API.

await car.enableApiControl();
await car.disableApiControl();

Note: When working with a Car instance, failure to disable the API after use may result in keyboard control of the car becoming unresponsive.

OK, let’s put this all together.

Step-1: Create the drive.ts file with car control code

Create a file nameddrive.ts file in the root folder of our project and add the following TypeScript code to it.

Notice the structure and flow of the code as this is a common structure of most AirSim-js programs.

You may notice the use of the Three.js Math classes, i.e.., Quaternion and Vector3. This is a well proven library that simplifies basic linear algebra mathematics.

Step-2: Transpile drive.ts

Convert the drive.ts file to the runnable drive.js JavaScript file.

npx tsc

Step-3: Run drive.js

Launch the Nodejs runtime with the drive.js file and watch the car drive in a circle.

node drive.js

Use the F1 key to access a menu to adjust your camera views as shown below.

Wrapping Up

Thanks for checking out this AirSim-js introduction. If you liked this article please give it a Thumbs Up and a share on twitter. You can reach me on Twitter at @ros2jsguy and find code that I’ve developed or working on at my GitHub account: https://github.com/ros2jsguy

Lastly, you can find additional AirSim-js examples in the AirSim-js GitHub repo. In the next article we’ll learn how to programmatically fly a drone in the AirSimNH (neighborhood) environment. Until then Happy Coding!

References

--

--

ROS2JsGuy

“ROS 2 for JavaScript & TypeScript Developers” tutorials from a guy named Wayne