Robot App Store

Robot App™ Description

The LEGO Street View Car app is controlled manually using a PSPNx sensor to receive commands from any standard PSP controller.
The PSPNx sensor has the advantage of being simple to program against and supports both analog joystick input for steering and boolean button presses to activate the camera. Press the triangle button on the PSP controller to capture an image and log the GPS coordinates.

Every time an image is captured the current latitude and longitude are recorded from the dGPS. The NXT creates a KML format file in the flash filesystem which is then uploaded from the NXT to a PC. Opening the KML file in Google Earth shows the path that the car drove, and also has placemarks for every picture you took along the way. Click on the placemark to see the picture.

For each GPS coordinate I create a KML Placemark entry that embeds descriptive HTML code using the CDATA tag. The image link in the HTML refers to the last image captured on disk. The 808 camera starts with the filename EKEN0001.jpg for the first picture captured. The second picture captured is EKEN0002.jpg, and so forth.

Developer: Mark C

Installation Instructions

Delivered as:

Service

The process flow to use the car is manual at the moment:
1. Attach the keyfob cameras to the PC using 4 USB cables
2. Delete all of the images on the cameras
3. Detach the cameras
4. Start the program on the NXT
5. Wait for the orange light to illuminate on the dGPS indicating a satellite fix is obtained.
6. The NXT display will show the current GPS coordinates.
7. Set the PSP controller to manual mode
8. Control the forward/reverse direction of driving with the left joystick, turn right/left with the right joystick.
9. Press the triangle button to take a picture and record the GPS coordinates
10. When done, press the orange button on the NXT.
11. Attach the NXT to the PC
12. Copy the Path.kml file from the NXT into an empty folder on the PC.
13. Attach each of the 4 USB cameras.
14. Take a group of 4 pictures across the cameras and merge into a panorama (STILL TO BE SOLVED).
15. Store the panorama into the folder containing the Path.kml
16. Open the Path.kml file in Google Earth and enjoy!

Robot App Requirements

Software

Robot-C

Hardware:

1. Chassis is based on the Land Rover chassis made by Sariel. Pendular suspension, four wheel drive, rack-and-pinion steering on front wheels only. (Requires LEGO Technic kit)
2. One NXT controls entire car.
3. One dGPS from Dexter Industries to capture current coordinates.
4. One PSPNx sensor from Mindsensors to receive commands from a PSP controller.
5. One NXTBee sensor from Dexter Industries to transmit telemetry to a base station.
6. One PFMate from Mindsensors to control the PF motors.
7. Four 808 type 3 cameras ordered from China to capture the images around the car.
8. One NXT motor for steering.
9. One NXT motor to activate the shutter release mechanism on the cameras.
10. Two PF XL motors to drive the front and rear wheels independently.
11. One PF batter box, and one PF receiver.
12. Control program written in NXC on the NXT using John Hansen’s enhanced
13. 1.31 firmware, and NXC 1.2.1 r4 on Mac OS.

Other requirements:

Nice day, the car is not water-proof!

Usage Instructions

This project is still very much in the proof-of-concept phase. It is a prototype with a lot of rough edges. The keyfob cameras are very temperamental and difficult to mount into standard LEGO dimensions.

What works:
1. The chassis can drive over grass carrying the full weight of all the components and batteries. That is one major achievement!
2. The dGPS works well when outdoors
3. I can record GPS waypoints in KML format.
4. The keyfob cameras are securely mounted
5. The NXTBee will send telemetry (battery level, GPS coordinates) to a receiver.


Yet to be done:
1. Obviously there is no live streaming of video or pictures from the car to a receiver base-station. I have a design in mind to achieve this.
2. The keyfob cameras are very sensitive to incorrect placement. Aligning the tiny shutter button with a LEGO piece requires a lot of fine tuning. Not every camera fires concurrently. For now I can reliably capture images from the front-facing camera.
3. The process to copy the images from the car is manual and not yet automated.
4. I don’t have a way to stitch the pictures together into a panorama that you can explore in 3D. I can do a 2D panorama, but again it’s a manual process.
5. The car can’t drive itself - yet!


How do we get the pictures and KML file from the NXT and into Google Earth?
First of all we need to get all the data in one place. The KML file refers to the relative path of each image, so we can package the KML file and the images into a single directory.

An example of the output produced is shown below. In this test case I started indoors in my house and took a few pictures. As you can see the dGPS has trouble getting an accurate reading and so the pictures appear to be scattered around the map. I then drove the car outside and started to capture pictures as I drove. From Snapshot 10 onwards the images become more realistic based on where the car actually is.