PinholeModel

giant.camera_models.pinhole_model:

class giant.camera_models.pinhole_model.PinholeModel(intrinsic_matrix=None, focal_length=1.0, field_of_view=None, use_a_priori=False, misalignment=None, estimation_parameters='basic', kx=None, ky=None, px=None, py=None, n_rows=1, n_cols=1, temperature_coefficients=None, a1=None, a2=None, a3=None)[source]

Bases: CameraModel

This class provides an implementation of the pinhole camera model for projecting 3d points onto images.

The PinholeModel class is a subclass of CameraModel. This means that it includes implementations for all of the abstract methods defined in the CameraModel class. This also means that it can be used throughout GIANT as the primary camera model, including within the calibration subpackage. If this class is going to be used with the calibration subpackage, the user can set which parameters are estimated and which are held fixed by using the estimation_parameters keyword argument when creating an instance of the class or by adjusting the estimation_parameters instance variable on an instance of the class. The estimation_parameters input/attribute is a string or list of strings specifying which parameters to estimate. This means that estimation_parameters could be something like 'basic' which would indicate to estimate just the usual parameters, or something like ['focal_length', 'ky', 'px', 'py'] to estimate just the terms included in the list.

In addition to the standard set of methods for a CameraModel subclass, the PinholeModel class provides the following additional methods which may or may not be useful to some people:

Method

Use

get_projections()

computes the pinhole, image frame, and pixel locations of a 3D point

pixels_to_gnomic()

removes distortion from a point to get the corresponding pinhole location in units of distance

The Pinhole model also provides the following additional properties for easy getting/setting:

Property

Description

field_of_view

the diagonal field of view of the camera in units of degrees

focal_length

the diagonal focal length of the camera in units of distance

kx

\(k_x\), the inverse of the pixel pitch in the x direction in units of pixels/distance

ky

\(k_y\), the inverse of the pixel pitch in the y direction in units of pixels/distance

px

\(p_{x}\), the x axis pixel location of the principal point of the camera in units of pixels

py

\(p_{y}\), the y axis pixel location of the principal point of the camera in units of pixels

a1

\(a_1\), the linear coefficient for focal length dependent focal length

a2

\(a_2\), the quadratic coefficient for focal length dependent focal length

a3

\(a_3\), the cubic coefficient for focal length dependent focal length

intrinsic_matrix_inv

The inverse of the intrinsic matrix
Parameters:

  • intrinsic_matrix (Sequence | ndarray | None) – the intrinsic matrix for the camera as a numpy shape (2, 3) array. Note that this is overwritten if kx, ky, px, py are also specified.

  • focal_length (Real) – The focal length of the camera in units of distance.

  • field_of_view (Real | None) – The field of view of the camera in units of degrees.

  • use_a_priori (bool) – A flag to indicate whether to include the a priori state vector in the Jacobian matrix when performing a calibration

  • misalignment (Sequence | ndarray | None) – either a numpy array of shape (3,) or a list of numpy arrays of shape(3,) with each array corresponding to a single image (the list of numpy arrays is only valid when estimating multiple misalignments)

  • estimation_parameters (str | Sequence) – A string or list of strings specifying which model parameters to include in the calibration

  • kx (Real | None) – The inverse of the pixel pitch along the x axis in units of pixel/distance

  • ky (Real | None) – The inverse of the pixel pitch along the y axis in units of pixel/distance

  • px (Real | None) – the x component of the pixel location of the principal point in the image in units of pixels

  • py (Real | None) – the y component of the pixel location of the principal point in the image in units of pixels

  • temperature_coefficients (Sequence | ndarray | None) – The temperature polynomial coefficients as a length 3 Sequence

  • a1 (Real | None) – the linear coefficient of the focal length temperature dependence

  • a2 (Real | None) – the quadratic coefficient of the focal length temperature dependence

  • a3 (Real | None) – the cubic coefficient of the focal length temperature dependence

  • n_rows (int) – the number of rows of the active image array

  • n_cols (int) – the number of columns in the active image array

n_rows

The number of rows in the active pixel array for the camera

n_cols

The number of columns in the active pixel array for the camera

use_a_priori

This boolean value is used to determine whether to append the identity matrix to the Jacobian matrix returned by compute_jacobian() in order to include the current estimate of the camera model in the calibration process.

intrinsic_matrix

The 2x3 intrinsic matrix contains the conversion from unitless gnomic locations to a location in an image with units of pixels.

It is defined as

\[\begin{split}\mathbf{K} = \left[\begin{array}{ccc} k_x & 0 & p_x \\ 0 & k_y & p_y \end{array}\right] \end{split}\]
temperature_coefficients

The coefficients for the polynomial specifying the change in the focal length as a function of temperature.

misalignment

Contains either a single rotation vector representing the misalignment between the specified camera frame and the actual camera frame, or a list of rotation vectors representing the misalignments between the specified camera frame and the actual camera frame for each image.

Typically you should not interface with this attribute directly and allow other GIANT objects to handle it, because it can get complicated to ensure it is in-sync with the number of images under consideration

estimate_multiple_misalignments

This boolean value is used to determine whether multiple misalignments are being estimated/used per image.

If set to True then one misalignment is estimated for each image and used for each image when projecting through the camera model. When set to False then a single misalignment is estimated for all images and used for all images when projecting through the camera model. Typically the user shouldn’t be setting this attribute directly as it is automatically handled when setting the estimation_parameters attribute

important_attributes

A list specifying the important attributes the must be saved/loaded for this camera model to be completely reconstructed.

property estimation_parameters: List[str]

A list of strings containing the parameters to estimate when performing calibration with this model.

This list is used in the methods compute_jacobian() and apply_update() to determine which parameters are being estimated/updated. From the compute_jacobian() method, only columns of the Jacobian matrix corresponding to the parameters in this list are returned. In the apply_update() method, the update vector elements are assumed to correspond to the order expressed in this list.

Valid values for the elements of this list are shown in the following table. Generally, they correspond to attributes of this class, with a few convenient aliases that point to a collection of attributes.

Value

Description

'basic'

estimate focal length, ky, and a single misalignment term for all images between the camera attitude and the spacecraft’s attitude: \(\left[\begin{array}{ccc} f & k_y & \boldsymbol{\delta\theta} \end{array}\right]\)

'intrinsic'

estimate focal length, kx, ky, px, and py: \(\left[\begin{array}{ccccc} f & k_x & k_y & p_x & p_y \end{array}\right]\). Note that this will likely result in a rank-deficient matrix without an a priori covariance. Use 'basic intrinsic' instead.

'basic intrinsic'

estimate focal length and ky: \(\left[\begin{array}{cc} f & k_y \end{array}\right]\)

'focal_length'

the focal length of the camera: \(f\)

'kx'

inverse of the pixel pitch along the x axis: \(k_x\)

'ky'

inverse of the pixel pitch along the y axis: \(k_y\)

'px'

x location of the principal point in pixels: \(p_x\)

'py'

y location of the principal point in pixels: \(p_y\)

'a1'

the linear coefficient for a temperature dependent focal length: \(a_1\)

'a2'

the quadratic coefficient for a temperature dependent focal length: \(a_2\)

'a3'

the cubic coefficient for a temperature dependent focal length: \(a_3\)

'temperature dependence'

estimate 3 temperature dependence coefficients for the focal length a1, a2, a3: \(\left[\begin{array}{ccc} a_1 & a_2 & a_3 \end{array}\right]\)

'single misalignment'

estimate a single misalignment for all images: \(\boldsymbol{\delta\theta}\)

'multiple misalignments'

estimate a misalignment for each image: \(\left[\begin{array}{ccc}\boldsymbol{\delta\theta}_1 & \ldots & \boldsymbol{\delta\theta}_n \end{array}\right]\)

Note that it may not be possible to estimate all attributes simultaneously because this may result in a rank deficient matrix in the calibration process (for instance, without setting a priori weights, estimating 'focal_length', 'kx', and 'ky' together would result in a rank deficient matrix. Therefore, just because you can set something in this list doesn’t mean you should.

For more details about calibrating a camera model, see the calibration package for details.

property kx: float

The inverse of the pixel pitch along the x axis in units of pix/distance.

This is the conversion factor to convert from gnomic coordinates (in units of distance) to units of pixels. It corresponds to the [0, 0] component of the intrinsic matrix

property ky: float

The inverse of the pixel pitch along the y axis in units of pix/distance.

This is the conversion factor to convert from pinhole coordinates (in units of distance) to units of pixels. It corresponds to the [1, 1] component of the intrinsic matrix

property px: float

The x pixel location of the principal point of the camera.

The principal point of the camera is the point in the image where the distortion is zero (the point where the optical axis pierces the image). This corresponds to the [0, 2] component of the intrinsic matrix

property py: float

The y pixel location of the principal point of the camera.

The principal point of the camera is the point in the image where the distortion is zero (the point where the optical axis pierces the image). This corresponds to the [1, 2] component of the intrinsic matrix

property a1: float

The linear coefficient for the focal length temperature dependence

This is the first term in the temperature_coefficients array and is multiplied by the temperature.

property a2: float

The quadratic coefficient for the focal length temperature dependence

This is the second term in the temperature_coefficients array and is multiplied by the temperature squared.

property a3: float

The cubic coefficient for the focal length temperature dependence

This is the third term in the temperature_coefficients array and is multiplied by the temperature cubed.

property focal_length: float

The focal length for the camera expressed in units of distance

property intrinsic_matrix_inv: ndarray

The inverse of the intrinsic matrix.

The inverse of the intrinsic matrix is used to convert from units of pixels with an origin at the upper left corner of the image to units of distance with an origin at the principal point of the image.

the intrinsic matrix has an analytic inverse which is given by

\[\begin{split}\mathbf{K}^{-1} = \left[\begin{array}{ccc} \frac{1}{k_x} & 0 & \frac{-p_x}{k_x} \\ 0 & \frac{1}{k_y} & \frac{-p_y}{k_y} \end{array}\right]\end{split}\]
To convert from units of pixels to units of distance you would do::
>>> from giant.camera_models import PinholeModel
>>> model = PinholeModel(kx=5, ky=10, px=100, py=500)
>>> ((model.intrinsic_matrix_inv[:, :2]@[[1, 2, 300], [4, 5, 600]]).T + model.intrinsic_matrix_inv[:, 2]).T
array([[-19.8, -19.6, 40.]
       [-49.6, -49.5, 10.]])

Note

For the PinholeModel, this same functionality is available from pixels_to_gnomic(). In classes with a distortion model (like the rest of the classes in this module) however, the above code will give you distorted gnomic location, while the pixels_to_gnomic() will give you undistorted gnomic locations (true pinhole points).

Note

Since the PinholeModel class defines the intrinsic matrix as a \(2\times 3\) matrix this isn’t a formal inverse. To get the true inverse you need to append a row of [0, 0, 1] to both the intrinsic matrix and intrinsic matrix inverse.

property field_of_view: float

A radial field of view of the camera specified in degrees.

The field of view should be set to at least the half width diagonal field of view of the camera. The field of view is used when querying star catalogues.

The diagonal field of view is defined as

+-----------+
|          /|
|         / |
|        /  |
|      V/   |
|     O/    |
|    F/     |
|   */      |
|  2/       |
|  /        |
| /         |
|/          |
+-----------+

If you specify this parameter to be None, the field of view will be computed using the camera model if possible.

Summary of Methods

project_onto_image

This method transforms 3D points or directions expressed in the camera frame into the corresponding 2D image locations.

project_directions

This method transforms 3D directions expressed in the camera frame into the corresponding 2D image directions.

compute_jacobian

Calculates the Jacobian matrix for each observation in unit_vectors_camera for each parameter to be estimated as defined in the estimation_parameters attribute.

compute_pixel_jacobian

This method computes the Jacobian matrix \(\partial\mathbf{x}_P/\partial\mathbf{x}_C\) where \(\mathbf{x}_C\) is a vector in the camera frame that projects to \(\mathbf{x}_P\) which is the pixel location.

compute_unit_vector_jacobian

This method computes the Jacobian matrix \(\partial\mathbf{x}_C/\partial\mathbf{x}_P\) where \(\mathbf{x}_C\) is a vector in the camera frame that projects to \(\mathbf{x}_P\) which is the pixel location.

apply_update

This method takes in a delta update to the camera parameters (\(\Delta\mathbf{c}\)) and applies the update to the current instance in place.

pixels_to_unit

This method converts pixel image locations to unit vectors expressed in the camera frame.

undistort_pixels

This method computes undistorted pixel locations (gnomic/pinhole locations) for given distorted pixel locations according to the current model.

distort_pixels

A method that takes gnomic pixel locations in units of pixels and applies the appropriate distortion to them.

overwrite

This method replaces self with the properties of model in place.

distortion_map

This method computes the value of the distortion model across an entire image for use in creating distortion maps.

undistort_image

This method takes in an entire image and warps it to remove the distortion specified by the current model.

copy

Returns a deep copy of this object, breaking all references with self.

to_elem

Stores this camera model in an etree._Element object for storing in a GIANT xml file

from_elem

This class method is used to construct a new instance of cls from an etree._Element object

adjust_temperature

This method adjusts a pixel location to reflect a new image temperature.

get_temperature_scale

This method computes the scaling to the focal length caused by a shift in temperature.

get_projections

This method computes and returns the pinhole, and pixel locations for a set of 3D points expressed in the camera frame.

pixels_to_gnomic

This method takes an input in pixels and computes the undistorted gnomic location in units of distance.

prepare_interp

This method prepares a SciPy RegularGridInterpolator for converting pixels into undistorted gnomic locations.

pixels_to_gnomic_interp

This method takes an input in pixels and approximates the undistorted gnomic location in units of distance.

get_state_labels

Convert a list of estimation parameters into state label names.

reset_misalignment

This method reset the misalignment terms to all be zero (no misalignment).

get_misalignment

This method returns the Rotation object for the misalignment for the requested image.