Donkey Simulator

The Donkey Gym project is a OpenAI gym wrapper around the Self Driving Sandbox donkey simulator (sdsandbox). When building the sim from source, checkout the donkey branch of the sdsandbox project.

The simulator is built on the the Unity game platform, uses their internal physics and graphics, and connects to a donkey Python process to use our trained model to control the simulated Donkey.

Installation Video:

Here's some videos to help you through the installation.



My Virtual Donkey

There are many ways to use the simulator, depending on your goals. You can use the simulator to get to know and use the standard Donkeycar drive/train/test cycle by treating it as virtual hardware. You will collect data, drive, and train using the same commands as if you were using a real robot. We will walk through that use-case first.



  • Download and unzip the simulator for your host pc platform from Donkey Gym Release.
  • Place the simulator where you like. For this example it will be ~/projects/DonkeySimLinux. Your dir will have a different name depending on platform.
  • Complete all the steps to install Donkey on your host pc.
  • Setup DonkeyGym:
cd ~/projects
git clone
cd gym-donkeycar
conda activate donkey
pip install -e .[gym-donkeycar]
  • You may use an existing ~/mycar donkey application, or begin a new one. Here we will start fresh:
donkey createcar --path ~/mysim
cd ~/mysim
  • Edit your to enable donkey gym simulator wrapper, replace <user-name> and the other parts of the path:
DONKEY_SIM_PATH = "/home/<user-name>/projects/DonkeySimLinux/donkey_sim.x86_64"
DONKEY_GYM_ENV_NAME = "donkey-generated-track-v0"

Note: your path to the executable will vary depending on platform and user. Windows: DonkeySimWin/donkey_sim.exe Mac OS: DonkeySimMac/ Linux: DonkeySimLinux/donkey_sim.x86_64


You may use all the normal commands to at this point. Such as:

python drive

This should start the simulator and connect to it automatically. By default you will have a web interface to control the donkey. Navigate to http://localhost:8887/drive to see control page.

On Ubuntu Linux only, you may plug in your joystick of choice. If it mounts as /dev/input/js0 then there's a good chance it will work. Modify to indicate your joystick model and use the --js arg to run.

python drive --js

As you drive, this will create a tub of records in your data dir as usual.


You will not need to rsync your data, as it was recorded and resides locally. You can train as usual:

python train --model models/mypilot.h5


You can use the model as usual:

python drive --model models/mypilot.h5

Then navigate to web control page. Set Mode and Pilot to Local Pilot(d). The car should start driving.

Sample Driving Data

Here's some sample driving data to get you started. Download this and unpack it into your data dir. This should train to a slow but stable driver.


Here's some info on the api to talk to the sim server. Make a TCP client and connect to port 9091 on whichever host the sim is running. The server sends and receives UTF-8 encoded JSON packets. Each message must have a "msg_type" field. The sim will end all JSON packets with a newline character for termination. You don't have to end each packet with a newline when sending to the server. But if it gets too many messages too quickly it may have troubles. Check the player log file for JSON parse errors if you are having troubles.

Get Protocol Version

Client=>Sim. Ask for the version of the protocol. Will help know when changes are made to these messages.

Fields: None


    "msg_type" : "get_protocol_version" 

Protocol Version

Sim=>Client. Reply for the version of the protocol. Currently at version 2.


  • version : string integer


    "msg_type" : "protocol_version",
    "version" : "2",

Scene Selection Ready

Sim=>Client. When the Menu scene is finished loading this will be sent. After this point, the sim can honor the Scene Loading message. (Menu only)

Fields: None


    "msg_type" : "scene_selection_ready" 

Get Scene Names

Client=>Sim. Ask names of the scene you can load. (Menu only)

Fields: None


    "msg_type" : "get_scene_names" 

Scene Names

Sim=>Client. Sim will reply with list of scene names.


  • scene_names : array of scene names


    "msg_type" : "scene_names" 
    "scene_names" : [ "generated_road", "warehouse", "sparkfun_avc". "generated_track" ]

Load Scene

Client=>Sim. Asks the sim to load one of the scenes from the Menu screen. (Menu only)


scene_name : generated_road | warehouse | sparkfun_avc | generated_track ( or whatever list the sim returns from get_scene_names)


        "msg_type" : "load_scene",
        "scene_name" : "generated_track"

Scene Loaded

Sim=>Client. Once scene is loaded, in reply, you will get a:

        "msg_type" : "scene_loaded"

Car Loaded

Sim=>Client. Once the sim finishes loading your car, it sends this message. The car is loaded for you automatically once the scene is loaded with an active client. Or a client make a connection.

Fields: None


    "msg_type" : "car_loaded" 

Car Config

Client=>Sim. Once loaded, you may configure your car visual details (scene only)


  • body_style : donkey | bare | car01 | cybertruck | f1
  • body_r : string value of integer between 0-255
  • body_g : string value of integer between 0-255
  • body_b : string value of integer between 0-255
  • car_name : string value car name to display over car. Newline accepted for multi-line.
  • font_size : string value of integer between 10-100 to set size of car name text


        "msg_type" : "car_config",
        "body_style" : "car01",
        "body_r" : "128",
        "body_g" : "0",
        "body_b" : "255",
        "car_name" : "Your Name",
        "font_size" : "100"

Camera Config

Client=>Sim. Once the scene is loaded, you may configure your car camera sensor details


  • fov : string value of float between 10-200. Sets the camera field of view in degrees.
  • fish_eye_x : string value of float between 0-1. Causes distortion warping in x axis.
  • fish_eye_y : string value of float between 0-1. Causes distortion warping in y axis.
  • img_w : string value of integer between 16-512. Sets camera sensor image width.
  • img_h : string value of integer between 16-512. Sets camera sensor image height.
  • img_d : string value of integer 1 or 3. Sets camera sensor image depth. In case of 1, you get 3 channels but all identicle with greyscale conversion done on the sim.
  • img_enc : Image format of data JPG | PNG | TGA
  • offset_x : string value of float. Moves the camera left and right axis.
  • offset_y : string value of float. Moves the camera up and down.
  • offset_z : string value of float. Moves the camera forward and back.
  • rot_x : string value of float. Degrees. Rotates camera around X axis.


    "msg_type" : "cam_config",
    "fov" : "150", 
    "fish_eye_x" : "1.0",
    "fish_eye_y" : "1.0",
    "img_w" : "255",
    "img_h" : "255",
    "img_d" : "1",
    "img_enc" : "PNG",
    "offset_x" : "0.0",
    "offset_y" : "3.0",
    "offset_z" : "0.0",
    "rot_x" : "90.0"

Note: You can add an other camera by changing the msg_type to "cam_config_b"

Control Car

Client=>Sim. Control throttle and steering.


  • steering : string value of float between -1 to 1. Maps to full left or right, 16 deg from center.
  • throttle : string value of float between -1 to 1. Full forward or reverse torque to wheels.
  • brake : string value of float between 0 to 1.


    "msg_type" : "control",
    "steering" : "0.0",
    "throttle" : "0.3",
    "brake" : "0.0"


Sim=>Client. The sim sends this message containing camera image and details about vehicle state. These come at a regular rate set in the sim. Usually about 20 HZ.


  • steering_angle : Last steering applied. Why not just steering like control? idk.
  • throttle : Last throttle applied.
  • speed : magnitude of linear velocity.
  • image : a BinHex encoded binary image. Use
  • imageb : (optionnal) same as above but for the second camera
  • lidar : (optionnal) list of lidar points in the following format: {d: distanceToObject, rx: rayRotationX, ry: rayRotationY}
  • hit : name of the last object struck. Or None if no object hit.
  • accel_x : x acceleration of vehicle.
  • accel_y : y acceleration of vehicle.
  • accel_z : z acceleration of vehicle.
  • gyro_x : x gyro acceleration.
  • gyro_y : y gyro acceleration.
  • gyro_z : z gyro acceleration.
  • gyro_w : w gyro acceleration.
  • pitch : pitch of the car in degrees.
  • roll : roll of the car degrees.
  • yaw : yaw of the car degrees.
  • activeNode : Progress on track (not working properly with multiple car for the moment)
  • totalNodes : number of nodes on track
  • pos_x : (training only) x world coordinate of vehicle.
  • pos_y : (training only) y world coordinate of vehicle.
  • pos_z : (training only) z world coordinate of vehicle.
  • vel_x : (training only) x velocity of vehicle.
  • vel_y : (training only) y velocity of vehicle.
  • vel_z : (training only) z velocity of vehicle.
  • cte : (training only) Cross track error. The distance from the car to the path in the center of the right most lane or center of the track (depends on the track)


    "msg_type" : "telemetry", 
    "steering_angle" : "0.0", 
    "throttle" : "0.0", 
    "speed" : "1.0", 
    "image" : "0x123...", 
    "hit" : "None", 
    "pos_x" : "0.0", 
    "pos_y" : "0.0", 
    "pos_z" : "0.0", 
    "accel_x" : "0.0", 
    "accel_y" : "0.0", 
    "accel_z" : "0.0", 
    "gyro_x" : "0.0", 
    "gyro_y" : "0.0", 
    "gyro_z" : "0.0", 
    "gyro_w" : "0.0",
    "pitch" : "0.0", 
    "roll" : "0.0", 
    "yaw" : "0.0",
    "activeNode" : "5"
    "totalNodes" : "26"
    "cte" : "0.5"

Reset Car

Client=>Sim. Return the car to the start point.

Fields: None


    "msg_type" : "reset_car" 

Set Car Position

Client=>Sim. Move the car to the given position (training only)

Fields: * pos_x : x world coordinate. * pos_y : y world coordinate. * pos_z : z world coordinate. * qx : (optionnal) quaternion x * qy : (optionnal) quaternion y * qz : (optionnal) quaternion z * qw : (optionnal) quaternion w


    "msg_type" : "set_position" 
    "pos_x" : "0.0", 
    "pos_y" : "0.0", 
    "pos_z" : "0.0"


    "msg_type" : "set_position" 
    "pos_x" : "0.0", 
    "pos_y" : "0.0", 
    "pos_z" : "0.0",
    "qx" : "0.0",
    "qy" : "0.2",
    "qz" : "0.0",
    "qw" : "1.0"

Get node position and rotation

Client=>Sim. Ask for a node_position packet


  • index : node index


    "msg_type": "node_position",
    "index": "0"

Node position and rotation

Sim=>Client. node_position packet (received after sending a node_position packet)


Fields: * pos_x : x world coordinate. * pos_y : y world coordinate. * pos_z : z world coordinate. * qx : (optionnal) quaternion x * qy : (optionnal) quaternion y * qz : (optionnal) quaternion z * qw : (optionnal) quaternion w


    "msg_type": "node_position",
    "Qx": "0",
    "Qy": "0",
    "Qz": "0",
    "Qw": "1",
    "pos_x": "0",
    "pos_y": "0",
    "pos_z": "0"

Exit Scene

Client=>Sim. Leave the scene and return to the main menu screen.

Fields: None


    "msg_type" : "exit_scene" 

Quit App

Client=>Sim. Close the sim executable. (Menu only)

Fields: None


    "msg_type" : "quit_app"