Performing Optical Navigation

With our camera calibration complete, we can now use GIANT to do some relative navigation! For this, we will process OpNav images taken during Dawn’s approach to the asteroid Vesta. These images were taken in sets of 40 for each OpNav window, alternating between long and short exposure lengths. The long exposure images are used to refine the attitude knowledge of the camera by observing stars in the image, while the short exposure images are used to perform the relative navigation with respect to Vesta.

To begin doing OpNav, create a script called opnav.py in the scripts directory and open it with your favorite text editor.

Initial Imports

As with the previous two scripts, we need to start off with importing the modules and packages we will need.

# a utility for retrieving a list of files using glob patterns
import glob

# the warning utility for filtering annoying warnings
import warnings

# a utility for generating plots
import matplotlib.pyplot as plt

# the Framing Camera object we defined before
from dawn_giant import DawnFCCamera, fc2_attitude, \
    vesta_attitude, vesta_position, sun_orientation, sun_position

# the function to load the camera model
from giant.camera_models import load

# The class we will use to perform the stellar opnav
from giant.stellar_opnav.stellar_class import StellarOpNav

# tool for visualizing the results of our star identification
from giant.stellar_opnav.visualizer import show_id_results

# the star catalogue we will use for our "truth" star locations
from giant.catalogues.giant_catalogue import GIANTCatalogue

# the class we will use to perform the relative navigation
from giant.relative_opnav.relnav_class import RelativeOpNav

# the point spread function for the camera
from giant.point_spread_functions.gaussians import Gaussian

# the scene we will use to describe how things are related spatially
from giant.ray_tracer.scene import Scene, SceneObject

# The shape object we will use for the sun
from giant.ray_tracer.shapes import Point

# the illumination function we will use to predict the image of our model
from giant.ray_tracer.illumination import McEwenIllumination

# some utilities from giant for visualizing the relative opnav results
from giant.relative_opnav.visualizer import limb_summary_gif, template_summary_gif, show_center_finding_residuals

# A module to provide access to the NAIF Spice routines
import spiceypy as spice

# python standard library serialization tool
import pickle

Loading the Data

With the imports out of the way, we now need to load our meta kernel, our images, and our camera model that we solved for in the calibration script and define the function that represents the point spread function for our camera. The loading is done using basic utilities from external libraries plus the load() function from the camera_models module. The PSF is defined using the OpenCV GaussianBlur function with some specified settings. These settings are camera dependent and are usually determined during the calibration campaign. For now, we’ll just use a rough guess for the PSF.

if __name__ == "__main__":
    # filter some annoying warnings
    warnings.filterwarnings('ignore', category=UserWarning)
    warnings.filterwarnings('ignore', category=DeprecationWarning)

    # furnish the meta kernel so we have all of the a priori state information
    spice.furnsh('./meta_kernel.tm')

    # choose the images we are going to process
    # use sorted to ensure they are in time sequential order
    images = sorted(glob.glob('../opnav/2011123_OPNAV_001/*.FIT') +
                    glob.glob('../opnav/2011165_OPNAV_007/*.FIT') +
                    glob.glob('../opnav/2011198_OPNAV_017/*.FIT'))

    # load the camera model we are using
    camera_model = load('dawn_camera_models.xml', 'FC2')

Loading the Images and Creating the Camera Instance

Now we can ask GIANT to load all of our images and create our camera instance. This is simply done by initializing our DawnFCCamera class as we did in the calibration script. The only difference now is that we also provide our PSF to the camera initializer so that GIANT knows about it.

# create the camera instance and load the images
camera = DawnFCCamera(images=images, model=camera_model, psf=Gaussian(sigma_x=0.75, sigma_y=0.75, size=5),
                      attitude_function=fc2_attitude)

Estimating the Rotation Using Star Images

With our camera object created, we can now start estimating the attitude in the long-exposure images using star observations. This is extremely similar to how we perform camera calibration, but we use the StellarOpNav class instead and we only estimate the attitude, not the calibration. Plus, we only want to look for stars in long exposure images so we tell GIANT to only use the long exposure images using the only_long_on() method of the StellarOpNav.camera attribute.

# do the stellar opnav to correct the attitude
# build the stellar opnav object, which is very similar to the calibration object but without the ability to do
# calibration.
sopnav = StellarOpNav(camera, star_id_kwargs={'catalogue': GIANTCatalogue()})

# ensure only the long exposure images are on
sopnav.camera.only_long_on()

# set the parameters to get a successful star identification
# we only need to estimate the attitude here so we can be fairly conservative
sopnav.star_id.max_magnitude = 8.0
sopnav.image_processing.poi_threshold = 20
sopnav.star_id.tolerance = 40
sopnav.star_id.ransac_tolerance = 1
sopnav.star_id.max_combos = 1000

# now id the stars and estimate the attitude
sopnav.id_stars()
sopnav.estimate_attitude()

# ensure we got a good id
show_id_results(sopnav)
sopnav.sid_summary()

If you run the script and save it you should see the id result plots appear (there will be a lot of them) and should see good results and post-fit residuals around 0.1 pixels in standard deviation. You can mess around with the various star identification and image processing parameters if you want or you can just leave them and move on. When you’re ready to move on then comment out the line with the show_id_results() function so that it doesn’t pop up every time we run the script.

Updating the Short Exposure Image Rotation

With the long exposure image attitudes corrected, we now want to use this information to update our short-exposure image attitudes. This is done in 2 steps. First, we turn on the only the short exposure images using the only_short_on() method. Then, we call the update_short_attitude() method which propagates the solved for attitudes in the long-exposure images to the following short-exposure image times using the attitude_function of the camera instance.

# now, we need to turn on the short exposure images, and use the updated attitude from the long exposure images to
# update the attitude for the short exposure images
sopnav.camera.only_short_on()
sopnav.camera.update_short_attitude()

Defining the OpNav Scene

Now that we have updated the attitude for the short-exposure images we need to define the OpNav scene. The OpNav scene tells GIANT what objects to expect in the images, as well as their relative position and orientation with respect to each other. For the DAWN approach to Vesta, we only have 3 objects we need to worry about in our scene: (1) the camera, (2) the sun, and (3) Vesta.

Lets begin by considering Vesta. For Vesta, we need a shape model which defines the terrain and shape of the body. GIANT uses the shape model when predicting what Vesta should look like in the field of view. To load the shape model, we use the pickle module from the python standard library to load the data from the kdtree.pickle file that we created when downloading our data. The kdtree.pickle contains a KDTree representation of the shape model that GIANT can understand and was created using the ingest_shape script that is packaged with GIANT.

# now we need to build our scene for the relative navigation.
# begin by loading the shape model
with open('../shape_model/kdtree.pickle', 'rb') as tree_file:

    vesta_shape = pickle.load(tree_file)

With the shape model loaded, we need to create an SceneObject instance for Vesta. The SceneObject class essentially wraps the shape model with functions that define its position and orientation in a scene at a given time, along with a name that GIANT can use to distinguish the object. In this case, the position and orientation functions we will use are wrappers to spice functions that we defined in our dawn_giant module before. The position function returns the positions of Vesta with respect to the Solar System Bary Center in the inertial frame. The orientation function returns the rotation from the Vesta fixed frame to the inertial frame as an Rotation object, which GIANT uses to rotate the shape model so that the correct side of the asteroid is viewed.

# we need to make this into a SceneObject, which essentially allows us to wrap the object with functions that
# give the state of the object at any given time
vesta_obj = SceneObject(vesta_shape, position_function=vesta_position, orientation_function=vesta_attitude, name='Vesta')

We also need to create a SceneObject for the sun. While the sun won’t be imaged directly (so we don’t need a shape model), we do need to know its relative position in the scene so that we can predict the illumination conditions. Therefore, we create a SceneObject wrapped around a Point object to represent the sun.

# now we need to form the SceneObject for our Sun Object
sun_obj = SceneObject(Point([0, 0, 0]), position_function=sun_position, orientation_function=sun_orientation)

Finally, we can define our actual scene. This is done by creating an Scene instance which includes our Vesta and Sun objects, as well as our camera instance which provides the scene relative information about the location and orientation of the camera in the inertial frame.

In this scene, Vesta is the only target we are observing, but GIANT is set up to allow multiple targets to be observed in the same scene, therefore we wrap the Vesta object in a list. The sun becomes the light source in the scene.

# now we can form our scene
opnav_scene = Scene(target_objs=[vesta_obj], light_obj=sun_obj)

Creating the RelNav Instance and extracting the observables

With the scene defined we can now create our RelativeOpNav instance. The RelativeOpNav class behaves very similarly to the StellarOpNav and Calibration classes, but exposes methods and settings for performing Relative Navigation instead of Stellar Navigation and Calibration.

We create the RelativeOpNav class by providing it the camera, the scene, a BRDF to translate viewing geometry into a predicted brightness, and a set of dictionaries to specify the settings for the various estimators in the RelNav class (these can also be set as attributes after initialization as with the StellarOpNav and Calibration classes).

The Vesta approach OpNavs only include images where Vesta is resolved (> 5 pixels in apparent diameter) thus we will only be using cross-correlation and only need to worry about settings for the XCorrCenterFinding class. In particular, we only really care about the grid_size and denoise_image settings. The grid_size setting specifies the number of rays we want to use to estimate the brightness in each pixel. GIANT always assumes a square grid and this number specifies the length of the sides. Therefore, if you specify a grid-size of 9, then you will use a 9x9 grid of rays for each pixel (which quickly adds up to a lot of rays). Because the body gets pretty large for our last day of OpNavs we are going to process, we’ll only use a grid_size of 3 pixels, which creates a 3x3 grid of rays for each pixel. The denoise_image flag specifies whether we want to attempt to decrease the noise in the image using a Gaussian Smoothing technique. Whether you set this flag to true or not depends on how noisey the images are. In general though, it is good to set this to True. We can also use the search_region setting to restrict how many pixels around the predicted location we should look for the correlation peak. This can be useful for images where the target is smaller in the field of view to ensure that we don’t get any false positives due to noise.

The brdf keyword argument to the RelativeOpNav class specifies the function that will convert viewing geometry (observation vector, illumination vector, surface normal, surface albedo) into a brightness value. GIANT has a number of BRDFs available in the illumination sub-module and in this case we’ll use the familiar McEwenIllumination BRDF.

# define the RelativeOpNav instance
# define the settings for the portions of Relnav
xcorr_kwargs = {"grid_size": 3, "denoise_image": True,
                'search_region': 50}

relnav = RelativeOpNav(camera, opnav_scene,
                       xcorr_kwargs=xcorr_kwargs,
                       brdf=McEwenIllumination(),
                       save_templates=True)

With the RelNav instance defined, we can now extract the observables, which take the form of observed pixel locations of the center-of-figure of the body in each image. We do this by calling the auto_estimate() method, which loops through each image, updates the scene to the predicted state at the time of the image, determines whether the body is resolved or not, and then locates the body in the image using either normalized cross correlation (resolved bodies) or by performing a Gaussian fit to the illumination data (unresolved bodies). Alternatively you could apply a specific relnav technique using ellipse_matching_estimate(), limb_matching_estimate(), cross_correlation_estimate(), moment_algorithm_estimate(), or unresolved_estimate(). You can try playing around with these if you want, though note that not all of the visualization routines will work with all of the methods.

relnav.auto_estimate()

And that is it, we’ve used GIANT to extract center-of-figure observables from real images of Dawn’s approach to Vesta. We can examine our results using the visualization functions we imported from GIANT. limb_summary_gif() creates a GIF showing the alignment of the limbs in each image after identifying the body, template_summary_gif() creates a GIF showing the actual image of the target and the predicted image of the target for each image and each target, and show_center_finding_residuals() shows the observed-computed center finding resiudals in pixels.

# show the results
limb_summary_gif(relnav)
template_summary_gif(relnav)
show_center_finding_residuals(relnav)
plt.show()

Note

If you receive an error about TypeError: 'NoneType' object is not callable then you likely need to update matplotlib by doing pip install --upgrade matplotlib

You can finish now, or you can try playing around with images from other OpNav days.

The Complete OpNav Script

For your convenience, the complete opnav.py script is presented here.

# a utility for retrieving a list of files using glob patterns
import glob

# the warning utility for filtering annoying warnings
import warnings

# a utility for generating plots
import matplotlib.pyplot as plt

# the Framing Camera object we defined before
from dawn_giant import DawnFCCamera, fc2_attitude, \
    vesta_attitude, vesta_position, sun_orientation, sun_position

# the function to load the camera model
from giant.camera_models import load

# The class we will use to perform the stellar opnav
from giant.stellar_opnav.stellar_class import StellarOpNav

# tool for visualizing the results of our star identification
from giant.stellar_opnav.visualizer import show_id_results

# the star catalogue we will use for our "truth" star locations
from giant.catalogues.giant_catalogue import GIANTCatalogue

# the class we will use to perform the relative navigation
from giant.relative_opnav.relnav_class import RelativeOpNav

# the point spread function for the camera
from giant.point_spread_functions.gaussians import Gaussian

# the scene we will use to describe how things are related spatially
from giant.ray_tracer.scene import Scene, SceneObject

# The shape object we will use for the sun
from giant.ray_tracer.shapes import Point

# the illumination function we will use to predict the image of our model
from giant.ray_tracer.illumination import McEwenIllumination

# some utilities from giant for visualizing the relative opnav results
from giant.relative_opnav.visualizer import limb_summary_gif, template_summary_gif, show_center_finding_residuals

# A module to provide access to the NAIF Spice routines
import spiceypy as spice

# python standard library serialization tool
import pickle


if __name__ == "__main__":
    # filter some annoying warnings
    warnings.filterwarnings('ignore', category=UserWarning)
    warnings.filterwarnings('ignore', category=DeprecationWarning)

    # furnish the meta kernel so we have all of the a priori state information
    spice.furnsh('./meta_kernel.tm')

    # choose the images we are going to process
    # use sorted to ensure they are in time sequential order
    images = sorted(glob.glob('../opnav/2011123_OPNAV_001/*.FIT') +
                    glob.glob('../opnav/2011165_OPNAV_007/*.FIT') +
                    glob.glob('../opnav/2011198_OPNAV_017/*.FIT'))

    # load the camera model we are using
    camera_model = load('dawn_camera_models.xml', 'FC2')

    # create the camera instance and load the images
    camera = DawnFCCamera(images=images, model=camera_model, psf=Gaussian(sigma_x=0.75, sigma_y=0.75, size=5),
                          attitude_function=fc2_attitude)

    # do the stellar opnav to correct the attitude
    # build the stellar opnav object, which is very similar to the calibration object but without the ability to do
    # calibration.
    sopnav = StellarOpNav(camera, star_id_kwargs={'catalogue': GIANTCatalogue()})

    # ensure only the long exposure images are on
    sopnav.camera.only_long_on()

    # set the parameters to get a successful star identification
    # we only need to estimate the attitude here so we can be fairly conservative
    sopnav.star_id.max_magnitude = 8.0
    sopnav.image_processing.poi_threshold = 20
    sopnav.star_id.tolerance = 40
    sopnav.star_id.ransac_tolerance = 1
    sopnav.star_id.max_combos = 1000

    # now id the stars and estimate the attitude
    sopnav.id_stars()
    sopnav.estimate_attitude()

    # ensure we got a good id
    # show_id_results(sopnav)
    sopnav.sid_summary()

    # now, we need to turn on the short exposure images, and use the updated attitude from the long exposure images to
    # update the attitude for the short exposure images
    sopnav.camera.only_short_on()
    sopnav.camera.update_short_attitude()

    # now we need to build our scene for the relative navigation.
    # begin by loading the shape model
    with open('../shape_model/kdtree.pickle', 'rb') as tree_file:

        vesta_shape = pickle.load(tree_file)

    # we need to make this into an SceneObject, which essentially allows us to wrap the object with functions that
    # give the state of the object at any given time
    vesta_obj = SceneObject(vesta_shape, position_function=vesta_position,
                            orientation_function=vesta_attitude, name='Vesta')

    # now we need to form the SceneObject for our Sun Object
    sun_obj = SceneObject(Point([0, 0, 0]), position_function=sun_position, orientation_function=sun_orientation)

    # now we can form our scene
    opnav_scene = Scene(target_objs=[vesta_obj], light_obj=sun_obj)

    # define the RelativeOpNav instance
    # define the settings for the portions of Relnav
    xcorr_kwargs = {"grid_size": 3, "denoise_image": True,
                    'search_region': 50}

    relnav = RelativeOpNav(camera, opnav_scene,
                           xcorr_kwargs=xcorr_kwargs,
                           brdf=McEwenIllumination(),
                           limb_matching_kwargs={'recenter': False},
                           save_templates=True)

    relnav.auto_estimate()

    # show the results
    limb_summary_gif(relnav)
    template_summary_gif(relnav)
    show_center_finding_residuals(relnav)
    plt.show()

Conclusion

And that’s the basics of GIANT. We successfully generated a camera model from star images and extracted center-of-figure observables from OpNav images for the DAWN approach to Vesta. There is certainly much more you can do with GIANT, but this provides a general overview of how things work and shows how you can quickly get GIANT working for a new mission. For more details, read through the rest of the documentation.