OpenCVModel¶
giant.camera_models.opencv_model
:
- class giant.camera_models.opencv_model.OpenCVModel(intrinsic_matrix=None, fx=None, fy=None, px=None, py=None, alpha=None, kx=None, ky=None, kxy=None, field_of_view=None, use_a_priori=False, distortion_coefficients=None, k1=None, k2=None, k3=None, radial2n=None, radial4n=None, radial6n=None, k4=None, k5=None, k6=None, radial2d=None, radial4d=None, radial6d=None, p1=None, p2=None, tiptilt_y=None, tiptilt_x=None, s1=None, s2=None, s3=None, s4=None, thinprism_1=None, thinprism_2=None, thinprism_3=None, thinprism_4=None, temperature_coefficients=None, a1=None, a2=None, a3=None, misalignment=None, estimation_parameters='basic', n_rows=1, n_cols=1)[source]¶
Bases:
BrownModel
This class provides an implementation of the OpenCV camera model for projecting 3D points onto images and performing camera calibration.
The OpenCV camera model is the pinhole camera model combined with a lens distortion model which projects any point along a ray emanating from the camera center (origin of the camera frame) to the same 2D point in an image. Given some 3D point (or direction) expressed in the camera frame, \(\mathbf{x}_C\), the OpenCV model is defined as
The
OpenCVModel
class is a subclasss ofCameraModel
. This means that it includes implementations for all of the abstract methods defined in theCameraModel
class. This also means that it can be used throughout GIANT as the primary camera model, including within thecalibration
subpackage. If this class is going to be used with thecalibration
subpackage, the user can set which parameters are estimated and which are held fixed by using theestimation_parameters
key word argument when creating an instance of the class or by adjusting theestimation_parameters
instance variable on an instance of the class. Theestimation_parameters
input/attribute is a string or list of strings specifying which parameters to estimate. This means thatestimation_parameters
could be something like'basic'
which would indicate to estimate just the usual parameters, or something like['focal_length', 'ky', 'px', 'py']
to estimate just the terms included in the list.In addition to the standard set of methods for a
CameraModel
subclass, theOpenCVModel
class provides the following additional methods which may or may not be useful to some people:Method
Use
computes the pinhole, image frame, and pixel locations of a 3D point
removes distortion from a point to get the corresponding unitless pinhole location
The
OpenCVModel
class also provides the following properties for easy getting/setting:Property
Description
the diagonal field of view of the camera in units of degrees
\(f_x\), focal length divided by the pixel pitch in the x direction in units of pixels
\(f_y\), focal length divided by the pixel pitch in the y direction in units of pixels
\(\alpha\), A alpha term for non-retangular pixels
\(p_{x}\), the x axis pixel location of the principal point of the camera in units of pixels
\(p_{y}\), the y axis pixel location of the principal point of the camera in units of pixels
\(k_1\), the radial distortion coefficient corresponding to \(r^2\) in the numerator
\(k_2\), the radial distortion coefficient corresponding to \(r^4\) in the numerator
\(k_3\), the radial distortion coefficient corresponding to \(r^6\) in the numerator
\(k_4\), the radial distortion coefficient corresponding to \(r^2\) in the denominator
\(k_5\), the radial distortion coefficient corresponding to \(r^4\) in the denominator
\(k_6\), the radial distortion coefficient corresponding to \(r^6\) in the denominator
\(p_1\), the tip/tilt/prism distortion coefficient corresponding to \(y_I\)
\(p_2\), the tip/tilt/prism distortion coefficient corresponding to \(x_I\)
\(s_1\), the first thin prism coefficient
\(s_1\), the second thin prism coefficient
\(s_1\), the third thin prism coefficient
\(s_1\), the fourth thin prism coefficient
\(a_1\), the linear coefficient for focal length dependent focal length
\(a_2\), the quadratic coefficient for focal length dependent focal length
\(a_3\), the cubic coefficient for focal length dependent focal length
The inverse of the intrinsic matrix
Note
The distortion attributes are aliases over each other and refer to the same data. Therefore setting a value to
radial2n
would also change the value ofk1
- Parameters:
intrinsic_matrix (Sequence | ndarray | None) – the intrinsic matrix for the camera as a numpy shape (2, 3) array. Note that this is overwritten if
kx
,ky
,kxy
,kyx
,px
,py
,fx
,fy
,alpha
are also specified.field_of_view (Real | None) – The field of view of the camera in units of degrees.
use_a_priori (bool) – A flag to indicate whether to include the a priori state vector in the Jacobian matrix when performing a calibration
distortion_coefficients (Sequence | ndarray | None) – A numpy array of shape (12,) containing the twelve distortion coefficients in order [k1, k2, k3, k4, k5, k6, p1, p2, s1, s2, s3, s4]. Note that this array is overwritten with any distortion coefficients that are specified independently.
misalignment (Sequence | ndarray | None) – either a numpy array of shape (3,) or a list of numpy arrays of shape(3,) with each array corresponding to a single image (the list of numpy arrays is only valid when estimating multiple misalignments)
estimation_parameters (str | Sequence) – A string or list of strings specifying which model parameters to include in the calibration
fx (Real | None) – The pixel pitch along the x axis in units of pixels
fy (Real | None) – The pixel pitch along the y axis in units of pixels
kx (Real | None) – The pixel pitch along the x axis in units of pixels
ky (Real | None) – The pixel pitch along the y axis in units of pixels
kxy (Real | None) – A alpha term for non-rectangular pixels
alpha (Real | None) – An alpha term for non-rectangular pixels
px (Real | None) – the x component of the pixel location of the principal point in the image in units of pixels
py (Real | None) – the y component of the pixel location of the principal point in the image in units of pixels
radial2n (Real | None) – the radial distortion coefficient corresponding to the r**2 term in the numerator
radial4n (Real | None) – the radial distortion coefficient corresponding to the r**4 term in the numerator
radial6n (Real | None) – the radial distortion coefficient corresponding to the r**4 term in the numerator
radial2d (Real | None) – the radial distortion coefficient corresponding to the r**2 term in the denominator
radial4d (Real | None) – the radial distortion coefficient corresponding to the r**4 term in the denominator
radial6d (Real | None) – the radial distortion coefficient corresponding to the r**4 term in the denominator
k1 (Real | None) – the radial distortion coefficient corresponding to the r**2 term in the numerator
k2 (Real | None) – the radial distortion coefficient corresponding to the r**4 term in the numerator
k3 (Real | None) – the radial distortion coefficient corresponding to the r**4 term in the numerator
k4 (Real | None) – the radial distortion coefficient corresponding to the r**2 term in the denominator
k5 (Real | None) – the radial distortion coefficient corresponding to the r**4 term in the denominator
k6 (Real | None) – the radial distortion coefficient corresponding to the r**4 term in the denominator
tiptilt_y (Real | None) – the tip/tilt/decentering distortion coefficient corresponding to the y term
tiptilt_x (Real | None) – the tip/tilt/decentering distortion coefficient corresponding to the x term
p1 (Real | None) – the tip/tilt/decentering distortion coefficient corresponding to the y term
p2 (Real | None) – the tip/tilt/decentering distortion coefficient corresponding to the x term
s1 (Real | None) – The first thin prism distortion coefficient
s2 (Real | None) – The second thin prism distortion coefficient
s3 (Real | None) – The third thin prism distortion coefficient
s4 (Real | None) – The fourth thin prism distortion coefficient
thinprism_1 (Real | None) – The first thin prism distortion coefficient
thinprism_2 (Real | None) – The second thin prism distortion coefficient
thinprism_3 (Real | None) – The third thin prism distortion coefficient
thinprism_4 (Real | None) – The fourth thin prism distortion coefficient
n_rows (int) – the number of rows of the active image array
n_cols (int) – the number of columns in the active image array
temperature_coefficients (Sequence | ndarray | None) –
a1 (Real | None) –
a2 (Real | None) –
a3 (Real | None) –
- n_rows¶
The number of rows in the active pixel array for the camera
- n_cols¶
The number of columns in the active pixel array for the camera
- use_a_priori¶
This boolean value is used to determine whether to append the identity matrix to the Jacobian matrix returned by
compute_jacobian()
in order to include the current estimate of the camera model in the calibration process.
- intrinsic_matrix¶
The 2x3 intrinsic matrix contains the conversion from unitless gnomic locations to a location in an image with units of pixels.
It is defined as
\[\begin{split}\mathbf{K} = \left[\begin{array}{ccc} f_x & \alpha & p_x \\ 0 & f_y & p_y \end{array}\right]\end{split}\]
- temperature_coefficients¶
The coefficients for the polynomial specifying the change in the focal length as a function of temperature.
- estimate_multiple_misalignments¶
This boolean value is used to determine whether multiple misalignments are being estimated/used per image.
If set to
True
then one misalignment is estimated for each image and used for each image when projecting through the camera model. When set toFalse
then a single misalignment is estimated for all images and used for all images when projecting through the camera model. Typically the user shouldn’t be setting this attribute directly as it is automatically handled when setting theestimation_parameters
attribute.
- estimation_parameters¶
A list of strings containing the parameters to estimate when performing calibration with this model.
This list is used in the methods
compute_jacobian()
andapply_update()
to determine which parameters are being estimated/updated. From thecompute_jacobian()
method, only columns of the Jacobian matrix corresponding to the parameters in this list are returned. In theapply_update()
method, the update vector elements are assumed to correspond to the order expressed in this list.Valid values for the elements of this list are shown in the following table. Generally, they correspond to attributes of this class, with a few convenient aliases that point to a collection of attributes.
Value
Description
'basic'
estimate fx, fy, alpha all 5 distortion parameters, and a single misalignment term for all images between the camera attitude and the spacecraft’s attitude: \(\left[\begin{array}{cccccccccc} f_x & f_y & \alpha & k_1 & k_2 & k_3 & p_1 & p_2 & \boldsymbol{\delta\theta} \end{array}\right]\)
'intrinsic'
estimate fx, fy, alpha, px, py, and all 5 distortion parameters: \(\left[\begin{array}{ccccccccccccc} f_x & f_y & \alpha & p_x & p_y & k_1 & k_2 & k_3 & p_1 & p_2 \end{array}\right]\)
'basic intrinsic'
estimate fx, fy, alpha, and all 5 distortion parameters: \(\left[\begin{array}{ccccccccc} f_x & f_y & \alpha & k1 & k_2 & k_3 & p_1 & p_2 \end{array}\right]\)
'temperature dependence'
estimate a 3rd order polynomial for temperature dependence: \(\left[\begin{array}{ccc} a_1 & a_2 & a_3 \end{array}\right]\)
'fx'
,'kx'
focal length divided by pixel pitch along the x axis: \(f_x\)
'alpha'
,'kxy'
alpha term for non-rectangular/rotated pixels: \(\alpha\)
'fy'
,'ky'
focal length divided by pixel pitch along the y axis: \(f_y\)
'px'
x location of the principal point: \(p_x\)
'py'
y location of the principal point: \(p_y\)
'radial2n'
,'k1'
distortion coefficient corresponding to the \(r^2\) term in the numerator: \(k_1\)
'radial4n'
,'k2'
distortion coefficient corresponding to the \(r^4\) term in the numerator: \(k_2\)
'radial6n'
,'k3'
distortion coefficient corresponding to the \(r^6\) term in the numerator: \(k_3\)
'radial2d'
,'k4'
distortion coefficient corresponding to the \(r^2\) term in the denominator: \(k_4\)
'radial4d'
,'k5'
distortion coefficient corresponding to the \(r^4\) term in the denominator: \(k_5\)
'radial6d'
,'k6'
distortion coefficient corresponding to the \(r^6\) term in the denominator: \(k_6\)
'tiptilt_y
,'p1'
distortion coefficient corresponding to the y tip/tilt distortion: \(p_1\)
'tiptilt_x
,'p2'
distortion coefficient corresponding to the x tip/tilt distortion: \(p_2\)
'thinprism_1'
,'s1'
first thin prism distortion coefficient: \(s_1\)
'thinprism_2'
,'s2'
second thin prism distortion coefficient: \(s_2\)
'thinprism_3'
,'s3'
third thin prism distortion coefficient: \(s_3\)
'thinprism_4'
,'s4'
fourth thin prism distortion coefficient: \(s_4\)
'a1'
estimate the linear coefficient for the temperature dependence: \(a_1\)
'a2'
estimate the quadratice coefficient for the temperature dependence: \(a_2\)
'a3'
estimate the cubic coefficient for the temperature dependence: \(a_3\)
'single misalignment'
estimate a single misalignment for all images: \(\boldsymbol{\delta\theta}\)
'multiple misalignments'
estimate a misalignment for each image: \(\left[\begin{array}{ccc}\boldsymbol{\delta\theta}_1 & \ldots & \boldsymbol{\delta\theta}_n \end{array}\right]\)
Note that it may not be possible to estimate all attributes simultaneously because this may result in a rank deficient matrix in the calibration process (for instance, without setting a priori weights, estimating
'px'
,'py'
, and'multiple misalignments'
together could result in a rank deficient matrix. Therefore, just because you can set something in this list doesn’t mean you should.For more details about calibrating a camera model, see the
calibration
package for details.
- distortion_coefficients¶
The distortion coefficients array contains the distortion coefficients for the OpenCV model [k1, k2, k3, k4, k5, k6, p1, p2, s1, s2, s3, s4]
- property field_of_view: float¶
A radial field of view of the camera specified in degrees.
The field of view should be set to at least the half width diagonal field of view of the camera. The field of view is used when querying star catalogues.
The diagonal field of view is defined as
+-----------+ | /| | / | | / | | V/ | | O/ | | F/ | | */ | | 2/ | | / | | / | |/ | +-----------+
If you specify this parameter to be
None
, the field of view will be computed using the camera model if possible.
- property k1: float¶
The radial distortion coefficient corresponding to the r**2 term in the numeratore
This corresponds to the [0] index of the distortion_coefficients array
- property kx: float¶
The inverse of the pixel pitch along the x axis in units of pix/distance.
This is the conversion factor to convert from gnomic coordinates (in units of distance) to units of pixels. It corresponds to the [0, 0] component of the intrinsic matrix
- property ky: float¶
The inverse of the pixel pitch along the y axis in units of pix/distance.
This is the conversion factor to convert from pinhole coordinates (in units of distance) to units of pixels. It corresponds to the [1, 1] component of the intrinsic matrix
- property fx: float¶
The focal length in units of pixels along the x axis (focal length divided by x axis pixel pitch)
This is an alias to
kx
and points to the (0, 0) index of the intrinsic matrix
- property fy: float¶
The focal length in units of pixels along the y axis (focal length divided by y axis pixel pitch)
This is an alias to
ky
and points to the (1, 1) index of the intrinsic matrix
- property alpha: float¶
An alpha term for non-rectangular pixels.
This is an alias to
kxy
and points to the to the [0, 1] component of the intrinsic matrix
- property kxy: float¶
An alpha term for non-rectangular pixels.
This corresponds to the [0, 1] component of the intrinsic matrix
- property px: float¶
The x pixel location of the principal point of the camera.
The principal point of the camera is the point in the image where the distortion is zero (the point where the optical axis pierces the image). This corresponds to the [0, 2] component of the intrinsic matrix
- property py: float¶
The y pixel location of the principal point of the camera.
The principal point of the camera is the point in the image where the distortion is zero (the point where the optical axis pierces the image). This corresponds to the [1, 2] component of the intrinsic matrix
- property a1: float¶
The linear coefficient for the focal length temperature dependence
This is the first term in the
temperature_coefficients
array and is multiplied by the temperature.
- property a2: float¶
The quadratic coefficient for the focal length temperature dependence
This is the second term in the
temperature_coefficients
array and is multiplied by the temperature squared.
- property a3: float¶
The cubic coefficient for the focal length temperature dependence
This is the third term in the
temperature_coefficients
array and is multiplied by the temperature cubed.
- property intrinsic_matrix_inv: ndarray¶
The inverse of the intrinsic matrix.
The inverse of the intrinsic matrix is used to convert from units of pixels with an origin at the upper left corner of the image to units of distance with an origin at the principal point of the image.
the intrinsic matrix has an analytic inverse which is given by
\[\begin{split}\mathbf{K}^{-1} = \left[\begin{array}{ccc} \frac{1}{f_x} & -\frac{\alpha}{f_xf_y} & \frac{\alpha p_y-f_yp_x}{f_xf_y} \\ 0 & \frac{1}{f_y} & \frac{-p_y}{f_y} \end{array}\right]\end{split}\]- To convert from units of pixels to a unitless, distorted gnomic location you would do::
>>> from giant.camera_models import BrownModel >>> model = BrownModel(kx=5, ky=10, px=100, py=500) >>> ((model.intrinsic_matrix_inv[:, :2]@[[1, 2, 300], [4, 5, 600]]).T + model.intrinsic_matrix_inv[:, 2]).T array([[-19.8, -19.6, 40.] [-49.6, -49.5, 10.]])
Note
The above code will give you distorted gnomic location, while the
pixels_to_gnomic()
will give you undistorted gnomic locations (true pinhole points).Note
Since the intrinsic matrix is defined as a \(2\times 3\) matrix this isn’t a formal inverse. To get the true inverse you need to append a row of [0, 0, 1] to both the intrinsic matrix and intrinsic matrix inverse.
- property k2: float¶
The radial distortion coefficient corresponding to the r**4 term in the numerator.
This corresponds to the [1] index of the distortion_coefficients array
- property k3: float¶
The radial distortion coefficient corresponding to the r**6 term in the numerator.
This corresponds to the [2] index of the distortion_coefficients array
- property k4: float¶
The radial distortion coefficient corresponding to the r**2 term in the denominator
This corresponds to the [3] index of the distortion_coefficients array
- property k5: float¶
The radial distortion coefficient corresponding to the r**4 term in the denominator
This corresponds to the [4] index of the distortion_coefficients array
- property k6: float¶
The radial distortion coefficient corresponding to the r**6 term in the denominator
This corresponds to the [5] index of the distortion_coefficients array
- property p1: float¶
The tip/tilt/decentering distortion coefficient corresponding to the y term.
This corresponds to the [6] index of the distortion_coefficients array.
- property p2: float¶
The tip/tilt/decentering distortion coefficient corresponding to the x term.
This corresponds to the [7] index of the distortion_coefficients array.
- property s1: float¶
The first thin prism distortion coefficient corresponding to r^2 for x.
This corresponds to the [8] index of the distortion_coefficients array
- property s2: float¶
The second thin prism distortion coefficient corresponding to r^4 for x.
This corresponds to the [9] index of the distortion_coefficients array
- property s3: float¶
The third thin prism distortion coefficient corresponding to r^2 for y.
This corresponds to the [10] index of the distortion_coefficients array
- property s4: float¶
The fourth thin prism distortion coefficient corresponding to r^4 for y.
This corresponds to the [11] index of the distortion_coefficients array
- property radial2n¶
The radial distortion coefficient corresponding to the r**2 term in the numeratore
This corresponds to the [0] index of the distortion_coefficients array
- property radial4n¶
The radial distortion coefficient corresponding to the r**4 term in the numerator.
This corresponds to the [1] index of the distortion_coefficients array
- property radial6n¶
The radial distortion coefficient corresponding to the r**6 term in the numerator.
This corresponds to the [2] index of the distortion_coefficients array
- property radial2d¶
The radial distortion coefficient corresponding to the r**2 term in the denominator
This corresponds to the [3] index of the distortion_coefficients array
- property radial4d¶
The radial distortion coefficient corresponding to the r**4 term in the denominator
This corresponds to the [4] index of the distortion_coefficients array
- property radial6d¶
The radial distortion coefficient corresponding to the r**6 term in the denominator
This corresponds to the [5] index of the distortion_coefficients array
- property tiptilt_y¶
The tip/tilt/decentering distortion coefficient corresponding to the y term.
This corresponds to the [6] index of the distortion_coefficients array.
- property tiptilt_x¶
The tip/tilt/decentering distortion coefficient corresponding to the x term.
This corresponds to the [7] index of the distortion_coefficients array.
- property thinprism_1¶
The first thin prism distortion coefficient corresponding to r^2 for x.
This corresponds to the [8] index of the distortion_coefficients array
- property thinprism_2¶
The second thin prism distortion coefficient corresponding to r^4 for x.
This corresponds to the [9] index of the distortion_coefficients array
- property thinprism_3¶
The third thin prism distortion coefficient corresponding to r^2 for y.
This corresponds to the [10] index of the distortion_coefficients array
- property thinprism_4¶
The fourth thin prism distortion coefficient corresponding to r^4 for y.
This corresponds to the [11] index of the distortion_coefficients array
Summary of Methods
This method transforms 3D points or directions expressed in the camera frame into the corresponding 2D image locations. |
|
This method transforms 3D directions expressed in the camera frame into the corresponding 2D image directions. |
|
Calculates the Jacobian matrix for each observation in unit_vectors_camera for each parameter to be estimated as defined in the |
|
This method computes the Jacobian matrix \(\partial\mathbf{x}_P/\partial\mathbf{x}_C\) where \(\mathbf{x}_C\) is a vector in the camera frame that projects to \(\mathbf{x}_P\) which is the pixel location. |
|
This method computes the Jacobian matrix \(\partial\mathbf{x}_C/\partial\mathbf{x}_P\) where \(\mathbf{x}_C\) is a vector in the camera frame that projects to \(\mathbf{x}_P\) which is the pixel location. |
|
This method takes in a delta update to camera parameters (\(\Delta\mathbf{c}\)) and applies the update to the current instance in place. |
|
This method converts pixel image locations to unit vectors expressed in the camera frame. |
|
This method computes undistorted pixel locations (gnomic/pinhole locations) for given distorted pixel locations according to the current model. |
|
A method that takes gnomic pixel locations in units of pixels and applies the appropriate distortion to them. |
|
This method replaces self with the properties of |
|
This method computes the value of the distortion model across an entire image for use in creating distortion maps. |
|
This method takes in an entire image and warps it to remove the distortion specified by the current model. |
|
Returns a deep copy of this object, breaking all references with |
|
Stores this camera model in an |
|
This class method is used to construct a new instance of cls from an |
|
This method adjusts a pixel location to reflect a new image temperature. |
|
This method computes the scaling to the focal length caused by a shift in temperature. |
|
This method takes an input in pixels and computes the undistorted gnomic location in units of distance. |
|
This method prepares a SciPy RegularGridInterpolator for converting pixels into undistorted gnomic locations. |
|
This method takes an input in pixels and approximates the undistorted gnomic location in units of distance. |
|
This method computes and returns the pinhole, and pixel locations for a set of 3D points expressed in the camera frame. |
|
This method applies the distortion model to the specified pinhole (gnomic) locations in the image frame. |
|
Convert a list of estimation parameters into state label names. |
|
This method reset the misalignment terms to all be zero (no misalignment). |
|
This method returns the Rotation object for the misalignment for the requested image. |