Recently I released AirSim-js, an open-source Nodejs client for AirSim (Aerial Informatics and Robotics Simulation). In this article I’ll introduce you to the cool AirSim environment and how to programmatically drive a car within AirSim using TypeScript and the AirSim-js package.
Background
If you’re new to AirSim, it’s an open-source 3D simulation platform developed by Microsoft AI Research. Out of the box it includes high resolution models of flying drones and driving cars within worlds such as a simple blocks world, a busy city street environment, a residential neighborhood and even an abandoned amusement park. Within these environments, vehicles, weather, time of day and simulation rate of execution are configurable. Another cool feature of AirSim is it’s support for hardware-in-the-loop. But for developers, the coolest feature is that AirSim can be programmatically controlled through an RPC interface based on the msgpack-rpc protocol.
You can see AirSim in action in this short video.
Vehicle Sensors
AirSim vehicles can be outfitted with multiple types of sensors. The supported sensors are:
- Barometer for altitude
- Distance meter
- GPS for position
- IMU for acceleration and angular velocity
- 3D LIDAR for point mapping
- Magnetometer for heading
The drone default sensors are: barometer, GPS, IMU and magnetometer. The default car sensors are the lonely GPS sensor. You can easily create a new drone or car with additional sensors such as LIDAR.
Cameras and Images
In addition to sensors, vehicles are preconfigured with 5 cameras and named by their position, e.g., front_center, back_center, bottom_center or fpv. The specifics of each camera are customizable. Beyond the default cameras, new cameras can added to a vehicle or installed externally anywhere in the sim world. From any camera you can request images in the following formats: RGB, distance, infrared, optical flow or segmentation.
ROS Integration
AirSim includes an integration with the increasingly popular Robot Operating System (ROS). This is a useful capability being able to tie AirSim into the ROS ecosystem of navigation and control algorithms, e.g. SLAM.
Getting Started with AirSim
Coding with AirSim-js is simple but first you’ll need to install AirSim and become familiar with it.
Install and Launch the AirSim Blocks environment
The AirSim project provides prebuilt Linux and Windows binaries for several simulation worlds.
- Begin by downloading the Blocks world for your platform (Linux or Windows).
2. Unzip the Blocks.zip download to a folder of your choice.
3. From a terminal, navigate to the Blocks executable and launch it in Car view mode.
#Linux
> cd install_dir/Blocks/LinuxNoEditor/Blocks/Blocks.sh#Windows
> cd install_dir\Blocks\WindowsNoEditor\Blocks\Binaries\Win64\Blocks.exe
On startup, AirSim will present the following view-mode selection dialog. Select Yes for Car view-mode.
The Blocks world window should appear containing a single car similar to this screenshot.
4. Practice driving the car manually using the Arrow keys on your keyboard.
Coding with AirSim-js
Ping the Blocks environment
Our 1st coding task is to run an AirSim rpc `ping` command against our running Blocks process. For this you will need to have Nodejs installed on your system and basic knowledge of TypeScript. See the following directions if you need assistance with this prerequisite step.
Step-1: Node-TypeScript project setup
Create and initialize a Nodejs TypeScript package. Begin by entering the following commands from a terminal:
mkdir airsim-js-intro
cd airsim-js-intro
npm init -y
npm i airsim-js
npm i typescript @types/node --save-dev
npx tsc --initNote — the Node-TypeScript project presented are for demonstration purposes only. Please don’t troll me.
Step-2: Coding ping.ts
Create a file named ping.ts and add the following TypeScript code to it. Review the comments for a narration of the code.
A special note about line-7. One of the responsibilities of the AirSim class is to serve as a factory for generic Vehicles, Cars and Multirotors. Therefore a class constructor of one of these vehicle classes is a required parameter. If you provide the constructor of Vehicle subclass such as Car or Multirotor it must match the view-mode of the AirSim world you are connecting. That is don’t run AirSim in Car mode and attempt to use the AirSim-js Multirotor API. Lastly, in the example, the code assumes that ping.ts be will executed on the same machine that is hosting the AirSim world. This is not a hard requriement as the AirSim constructor included parameters for a custom IP and port number.
For more AirSim class details see AirSim TSDoc.
/**
* Create an AirSim client.
* AirSim serves as a factory for Vehicles, (e.g., Car, Multirotor)
* hosted in an AirSim environment. When constructing a new AirSim
* client instance you must provide the class of Vehicle hosted by AirSim.
* @param vehicleClass — The type of vehicle hosted by the AirSim server
* @param port — The AirSim server port number, default = 41451
* @param ip — The AirSim server IP address, default = 127.0.0.1
*/
constructor(
private vehicleClass: Constructor<T>,
public port: string | number = 41451,
public readonly ip = '127.0.0.1'
)Step-3: Transpile ping.ts
This step converts the ping.ts file to the runnable ping.js javascript file.
npx tscStep-4: Run ping
Run the ping.js file using the Nodejs runtime.
node ping.jsIf everything works correctly the console output should be similar to this:
> node ping.js
connected: true
Calling ping
result: true
Connected!
Client Ver: 1 (Min Req: 1)
Server Ver: 1 (Min Req: 1)Drive the Car in the Blocks environment
Next let’s write some code to do something interesting like drive the car. Feel free to skip ahead to the code for driving the Blocks car.
The idea is to command the car’s steering angle and throttle such that the car turns in a circle. We will do that using a CarControls object. Here’s the definition of CarControls:
/**
* Car control settings
*/
export type CarControls = {
throttle: number, // 0.0–1.0
steering: number, // -1.0 to 1.0, left to right
brake: number, // 0.0–1.0
handbrake: boolean, // true = apply handbrake */
is_manual_gear: boolean, // true implies use manual_gear prop
manual_gear: number, // -1 to 5, 0 = neutral
gear_immediate: boolean // shift to manual gear setting now
}We can create a CarControls instance from scratch or request the current CarControls from a car instance. Then we modify the subset of properties that represent our intentions as shown.
const controls = await car.getControls();
controls.throttle = 0.25; // forward at 1/4 speed
controls.steering = 1.0; // full right turnGetting a handle to the Car
Before we can send commands to a car we need a handle to it. We can do this in 2 ways. The first is to access the car by name, assuming we know the car’s unique name. The second approach shown below is to request all the vehicles and find the target car in the list.
const cars = await airsim.getVehicles();
const car = cars[0]; // use the 1st carAPI Control — Enable before use; Disable when not in use
Before we can send commands to a vehicle we need to enable it’s API. Then upon completion of our use of the vehicle’s api, we should always disable the API.
await car.enableApiControl();
await car.disableApiControl();Note: When working with a Car instance, failure to disable the API after use may result in keyboard control of the car becoming unresponsive.
OK, let’s put this all together.
Step-1: Create the drive.ts file with car control code
Create a file nameddrive.ts file in the root folder of our project and add the following TypeScript code to it.
Notice the structure and flow of the code as this is a common structure of most AirSim-js programs.
You may notice the use of the Three.js Math classes, i.e.., Quaternion and Vector3. This is a well proven library that simplifies basic linear algebra mathematics.
Step-2: Transpile drive.ts
Convert the drive.ts file to the runnable drive.js JavaScript file.
npx tscStep-3: Run drive.js
Launch the Nodejs runtime with the drive.js file and watch the car drive in a circle.
node drive.jsUse the F1 key to access a menu to adjust your camera views as shown below.
Wrapping Up
Thanks for checking out this AirSim-js introduction. If you liked this article please give it a Thumbs Up and a share on twitter. You can reach me on Twitter at @ros2jsguy and find code that I’ve developed or working on at my GitHub account: https://github.com/ros2jsguy
Lastly, you can find additional AirSim-js examples in the AirSim-js GitHub repo. In the next article we’ll learn how to programmatically fly a drone in the AirSimNH (neighborhood) environment. Until then Happy Coding!
