Monday, November 24, 2014

Thursday, November 20, 2014

Python Sets to Align CSV Columns

From recent work, I've found Python set operations to be easier than using Excel VLOOKUP or MATCH formulas.  Here are a few uses:

Visually align two columns from a CSV file, adding the phrase "--MISSING--" into columns where values do not match.

SOMEFILE = '/path/to/some/file.csv'

with open(SOMEFILE, 'r') as csvfile:
                filereader = csv.reader(csvfile, delimiter=',',quotechar='"')
                for row in filereader:
                        set_old.add(row[0])  #column 0

                        set_new.add(row[1])  #column 1

total = sorted(set_old|set_new)
list1 = [x if x in set_old else "--MISSING--" for x in total]
list2 = [x if x in set_new else "--MISSING--" for x in total]

for i in range(0, len(list1)):
                print i, "\t", list1[i], "\t", list2[i]


You can follow this up and pipe the output to "column -t" command in linux to produce an easy to read list.

python script.py | column -t

For example

Column0        Column1
1                     2
2                     3

would become:

Column0        Column1
1                     --Missing--
2                      2
--Missing--      3

It is very easy to pull keys from an existing array or dictionary and perform set operations on them such as comparing lists of users in access or configuration files.

for user in new_users.difference(old_users):
         print user



Here are some other interesting set functions:

A = set(['1','2','3','4'])
B = set(['3','4','5','6'])
C = set(['1','2','3','4','5','6','7','8','9','10'])
D = set(['1','2'])

A.issubset(B)
False

A.issubset(C)
True

A.issuperset(B)
False

A.issuperset(D)
True

A.intersection(B)
['3','4']

A.union(B)
['1','2','3','4','5','6']

A.difference(B)
['1','2']

A.symmetric_difference.B
['1','2','5','6']

Have you used Python sets in Geospatial applications?  Let me know how in the comments.

https://docs.python.org/2/library/sets.html


Wednesday, November 12, 2014

Safety and the National Air Space

Update(11/24/2014)

FAA

The FAA may some day require some type of pilots license to commercially operate UAV safely in the National Air Space.  With legislation expected by the end of the year, it seems like a good idea to start reviewing the FAA knowledge tests.

https://www.faa.gov/training_testing/testing/test_guides/

Check out the Recreational Pilot and Private Knowledge Test.  This has resources covering airplanes, helicopters, gliders, balloons, and airships, all of which have useful information for safely and effectively operating a variety of aerial vehicles.

This resource has good information on learning the physics and prerequisite knowledge for flying.
http://www.faa.gov/regulations_policies/handbooks_manuals/aviation/pilot_handbook/

DJI Phantom 2 Vision Plus

While DJI Phantom supports "no fly zones" of 5 miles around airports, it is very easy to overlook that  this only includes class A and B airports. These are basically the large international airports.   Smaller airports are not included.  It is still possible to accidentally fly near an airport.

Before flying in unknown areas, I recommend exploring the area with Google or Bing Maps to check for the presence of airports and other potentially hazards nearby.  It's pretty easy to get swept up with the excitement of flying.

I did contact DJI about adding airports to the no fly zone and this is how they replied:

Dear DJI Customer,

Thank you for contacting DJI with your question about No Fly Zones.
Please see the websites below for quick reference. For further enquiries, please reply to this email with support@dji.com in CC, and we will get back to you as soon as possible.
Best Regards,
DJI Fly Safe Team



Tuesday, November 11, 2014

Galaxy S4 Photogrammetry

The Galaxy S4 is lightweight, loaded with sensors, and has plenty of power/battery life. The S4 is able to run the lightweight Seek Thermal Imaging camera as a USB attachment. With proper software, this device should make a great addition to many UAV.

Specs & Weight


 Camera Rear Focal Length 31mm
 Camera Rear Resolution 13 MP
 Camera Front Resolution  2 MP
 Location AGPS, GLONASS
 Weight 130.2g
 Size5.38" x 2.74" x 0.31"
 Android 4.4.2+
 RAM 2GB
 Processor Quadcore 1.9 Ghz
 Disk space 16 GB
 SensorsAccelerometer
Barometer
Geomagnetic
Gyrometer
Proximity
Temperature
Humidity

Project outline

Read in a flight plan from the Mission Planner software.

Log flight data to SD card.
Transmit sensor and/or video data over the DJI Phantom 2 Vision network via UDP to a laptop.
Use the sensor data to take photos with the 13 MP camera and the Seek Thermal camera simultaneously.
Aerial photography independent of flight apparatus.
Use coffee cans with internal fire as thermal ground control points.
Post processing with Visual SFM/CMPMVS to create orthorectified photos.

Getting Started

  • Install Android Developer Studio
  • Install Java SDK x64
  • Enable Developer Options on Android Phone. Do this by tapping the About Phone 7 times until this menu becomes available.
  • Install USB Driver for the Samsung Galaxy S4 SCH-I545
  • Develop. Make sure to save work in a Git repository such as Github.

Monday, November 10, 2014

Seek Thermal Imaging Camera

Just received my Seek Thermal Imaging Camera on 11/6/14.   Here are the specifications:

Spatial Resolution206 x 156 (32,136 pixels)
Misc12 mu pixel pitch
SensorVanadium Oxide Microbolometer
LensCholcogenide Lens
FOV36 degrees
Spectral Resolution7.2 - 13 microns
OSAndroid 4.3 or higher with mUSB
Temperature Range-40C to 330C
Price$200 USD
Weight14.1 g

Other features include:
  • Watermark images with GPS coordinates
  • Display temperature of an object
  • Adjust color pallets
  • Auto seek high and low temperatures
  • Thresholding above, equal to, or below a limit
Michigan Tech MEEM Building
Galaxy S4 with Seek Thermal Camera

My truck interior on a cold evening




Project Hardware and Software



Here is a summary of the software and hardware I've been working with recently.

Software


Autodesk Recap - This is cloud based processing of imagery and produces 3D models.

  • Cost: Free with Academic License 
  • Pros: Does not require hardware to process imagery 
  • Cons: Requires time to upload imagery 

Autodesk Revit - This is building information modeling software that can be used to make 3D layouts from the Recap output.

Agisoft Photoscan - This is a commercial product. There are two versions: Standard and Professional.

  • Cost: 600 - 3500 USD. Free demo but lacks ability to export results. 
  • Pros: Ease of use. Does a great job processing with default parameters. 
  • Cons: Requires lots of RAM. 16 GB should be minimum. 

VisualSFM/CMPMVS - This is an open source product. This software allows the creation of 3D Models, orthorectified imagery, digital surface models.
  • Cost: Free. 
  • Pros: Free. 
  • Cons: Requires lots of disk space and CPU. Can be intimidating. 

Adobe Lightroom - This was useful for the DJI Phantom lens correction profile.


  • Cost: 79 USD
  • Pros: Lens correction profile for DJI camera. Easy to use contrast and histogram editing.
  • Cons: Cost.


Meshlab - This is software for editing 3D models. It can be used to clean up artifact from your models

Hugin - This is a free open source software for stitching images together. It is often used for creating panoramic photos.


Microsoft ICE - Like Hugin, this is a free software (closed source) for stitching images together. It can create panoramic imagery that is compatible with Photosynth.


Three.JS - This is a Javascript framework for working with WebGL. It allows 3D visualization of the OBJ files.

RealFlight 7.5 - This is software used to practice flying. It has very realistic physics and supports many types of vehicles from tri/quad/hex copters to fixed wing aircraft and traditional helicopters.


Hardware

DJI Phantom 2 Vision Plus

  • Cost: 1700 USD including Pelican case, prop guards, and extra battery, charger. 
  • Pro: Ease of use. Resale value. FPV/Telemetry 
  • Con: Camera quality. Lack of mission planning software. 

Blade Nano QX

  • Cost: 90 USD 
  • Pro: Easy to fly. Can fly indoors. Has prop guards 
  • Con: None. This is an excellent physical training quadcopter. 

RealFlight 7.5

  • Cost: 179 USD 
  • Pro: Practice flying without damaging apparatus. Elite Interlink edition has a controller similar to Futaba radio. 
  • Con: Price. 

Canon SX260

  • Cost: 250 USD 
  • Pro: Canon CHDK. Image quality. Ability to remove infrared filter. 
  • Con: Weighs a hefty 230g. 

Samsung Galaxy S4

  • Cost: Used 75 USD. 
  • Pro: Sensors, weight, Android SDK. 
  • Con: None so far. 

Seek Thermal Imaging Camera

  • Cost 200 USD 
  • Pro: True longwave infrared thermal imaging in 7.5 - 13 micron range. 
  • Con: Resolution.