kwimage.util_warp

Module Contents

Functions

_coordinate_grid(dims, align_corners=False) Creates a homogenous coordinate system.
warp_image(inputs, mat, **kw)
warp_tensor(inputs, mat, output_dims, mode=’bilinear’, padding_mode=’zeros’, isinv=False, ishomog=None, align_corners=False, new_mode=False) A pytorch implementation of warp affine that works similarly to
subpixel_align(dst, src, index, interp_axes=None) Returns an aligned version of the source tensor and destination index.
subpixel_set(dst, src, index, interp_axes=None) Add the source values array into the destination array at a particular
subpixel_accum(dst, src, index, interp_axes=None) Add the source values array into the destination array at a particular
subpixel_maximum(dst, src, index, interp_axes=None) Take the max of the source values array into and the destination array at a
subpixel_minimum(dst, src, index, interp_axes=None) Take the min of the source values array into and the destination array at a
subpixel_slice(inputs, index) Take a subpixel slice from a larger image. The returned output is
subpixel_translate(inputs, shift, interp_axes=None, output_shape=None) Translates an image by a subpixel shift value using bilinear interpolation
_padded_slice(data, in_slice, ndim=None, pad_slice=None, pad_mode=’constant’, **padkw) Allows slices with out-of-bound coordinates. Any out of bounds coordinate
_ensure_arraylike(data, n=None)
_rectify_slice(data_dims, low_dims, high_dims, pad_slice=None) Given image dimensions, bounding box dimensions, and a padding get the
_warp_tensor_cv2(inputs, mat, output_dims, mode=’linear’, ishomog=None) implementation with cv2.warpAffine for speed / correctness comparison
warp_points(matrix, pts, homog_mode=’divide’) Warp ND points / coordinates using a transformation matrix.
remove_homog(pts, mode=’divide’) Remove homogenous coordinate to a point array.
add_homog(pts) Add a homogenous coordinate to a point array
subpixel_getvalue(img, pts, coord_axes=None, interp=’bilinear’, bordermode=’edge’) Get values at subpixel locations
subpixel_setvalue(img, pts, value, coord_axes=None, interp=’bilinear’, bordermode=’edge’) Set values at subpixel locations
_bilinear_coords(ptsT, impl, img, coord_axes)
kwimage.util_warp.TORCH_GRID_SAMPLE_HAS_ALIGN
kwimage.util_warp._coordinate_grid(dims, align_corners=False)

Creates a homogenous coordinate system.

Parameters:
  • dims (Tuple[int]*) – height / width or depth / height / width
  • align_corners (bool) – returns a grid where the left and right corners assigned to the extreme values and intermediate values are interpolated.
Returns:

Tensor[shape=(3, *DIMS)]

References

https://github.com/ClementPinard/SfmLearner-Pytorch/blob/master/inverse_warp.py

Example

>>> # xdoctest: +IGNORE_WHITESPACE
>>> _coordinate_grid((2, 2))
tensor([[[0., 1.],
         [0., 1.]],
        [[0., 0.],
         [1., 1.]],
        [[1., 1.],
         [1., 1.]]])
>>> _coordinate_grid((2, 2, 2))
>>> _coordinate_grid((2, 2), align_corners=True)
tensor([[[0., 2.],
         [0., 2.]],
        [[0., 0.],
         [2., 2.]],
        [[1., 1.],
         [1., 1.]]])
kwimage.util_warp.warp_image(inputs, mat, **kw)
kwimage.util_warp.warp_tensor(inputs, mat, output_dims, mode='bilinear', padding_mode='zeros', isinv=False, ishomog=None, align_corners=False, new_mode=False)

A pytorch implementation of warp affine that works similarly to cv2.warpAffine / cv2.warpPerspective.

It is possible to use 3x3 transforms to warp 2D image data. It is also possible to use 4x4 transforms to warp 3D volumetric data.

Parameters:
  • inputs (Tensor[…, *DIMS]) – tensor to warp. Up to 3 (determined by output_dims) of the trailing space-time dimensions are warped. Best practice is to use inputs with the shape in [B, C, *DIMS].

  • mat (Tensor) – either a 3x3 / 4x4 single transformation matrix to apply to all inputs or Bx3x3 or Bx4x4 tensor that specifies a transformation matrix for each batch item.

  • output_dims (Tuple[int]*) –

    The output space-time dimensions. This can either be in the form

    (W,), (H, W), or (D, H, W).

  • mode (str) – Can be bilinear or nearest. See torch.nn.functional.grid_sample

  • padding_mode (str) – Can be zeros, border, or reflection. See torch.nn.functional.grid_sample.

  • isinv (bool, default=False) – Set to true if mat is the inverse transform

  • ishomog (bool, default=None) – Set to True if the matrix is non-affine

  • align_corners (bool, default=False) – Note the default of False does not work correctly with grid_sample in torch <= 1.2, but using align_corners=True isnt typically what you want either. We will be stuck with buggy functionality until torch 1.3 is released.

    However, using align_corners=0 does seem to reasonably correspond with opencv behavior.

Notes

Also, it may be possible to speed up the code with F.affine_grid

KNOWN ISSUE: There appears to some difference with cv2.warpAffine when
rotation or shear are non-zero. I’m not sure what the cause is. It may just be floating point issues, but Im’ not sure.

Todo

  • [ ] FIXME: see example in Mask.scale where this algo breaks when

the matrix is 2x3 - [ ] Make this algo work when matrix ix 2x2

References

https://discuss.pytorch.org/t/affine-transformation-matrix-paramters-conversion/19522 https://github.com/pytorch/pytorch/issues/15386

Example

>>> # Create a relatively simple affine matrix
>>> import skimage
>>> mat = torch.FloatTensor(skimage.transform.AffineTransform(
>>>     translation=[1, -1], scale=[.532, 2],
>>>     rotation=0, shear=0,
>>> ).params)
>>> # Create inputs and an output dimension
>>> input_shape = [1, 1, 4, 5]
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> output_dims = (11, 7)
>>> # Warp with our code
>>> result1 = warp_tensor(inputs, mat, output_dims=output_dims, align_corners=0)
>>> print('result1 =\n{}'.format(ub.repr2(result1.cpu().numpy()[0, 0], precision=2)))
>>> # Warp with opencv
>>> import cv2
>>> cv2_M = mat.cpu().numpy()[0:2]
>>> src = inputs[0, 0].cpu().numpy()
>>> dsize = tuple(output_dims[::-1])
>>> result2 = cv2.warpAffine(src, cv2_M, dsize=dsize, flags=cv2.INTER_LINEAR)
>>> print('result2 =\n{}'.format(ub.repr2(result2, precision=2)))
>>> # Ensure the results are the same (up to floating point errors)
>>> assert np.all(np.isclose(result1[0, 0].cpu().numpy(), result2, atol=1e-2, rtol=1e-2))

Example

>>> # Create a relatively simple affine matrix
>>> import skimage
>>> mat = torch.FloatTensor(skimage.transform.AffineTransform(
>>>     rotation=0.01, shear=0.1).params)
>>> # Create inputs and an output dimension
>>> input_shape = [1, 1, 4, 5]
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> output_dims = (11, 7)
>>> # Warp with our code
>>> result1 = warp_tensor(inputs, mat, output_dims=output_dims)
>>> print('result1 =\n{}'.format(ub.repr2(result1.cpu().numpy()[0, 0], precision=2, supress_small=True)))
>>> print('result1.shape = {}'.format(result1.shape))
>>> # Warp with opencv
>>> import cv2
>>> cv2_M = mat.cpu().numpy()[0:2]
>>> src = inputs[0, 0].cpu().numpy()
>>> dsize = tuple(output_dims[::-1])
>>> result2 = cv2.warpAffine(src, cv2_M, dsize=dsize, flags=cv2.INTER_LINEAR)
>>> print('result2 =\n{}'.format(ub.repr2(result2, precision=2)))
>>> print('result2.shape = {}'.format(result2.shape))
>>> # Ensure the results are the same (up to floating point errors)
>>> # NOTE: The floating point errors seem to be significant for rotation / shear
>>> assert np.all(np.isclose(result1[0, 0].cpu().numpy(), result2, atol=1, rtol=1e-2))

Example

>>> # Create a random affine matrix
>>> import skimage
>>> rng = np.random.RandomState(0)
>>> mat = torch.FloatTensor(skimage.transform.AffineTransform(
>>>     translation=rng.randn(2), scale=1 + rng.randn(2),
>>>     rotation=rng.randn() / 10., shear=rng.randn() / 10.,
>>> ).params)
>>> # Create inputs and an output dimension
>>> input_shape = [1, 1, 5, 7]
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> output_dims = (3, 11)
>>> # Warp with our code
>>> result1 = warp_tensor(inputs, mat, output_dims=output_dims, align_corners=0)
>>> print('result1 =\n{}'.format(ub.repr2(result1.cpu().numpy()[0, 0], precision=2)))
>>> # Warp with opencv
>>> import cv2
>>> cv2_M = mat.cpu().numpy()[0:2]
>>> src = inputs[0, 0].cpu().numpy()
>>> dsize = tuple(output_dims[::-1])
>>> result2 = cv2.warpAffine(src, cv2_M, dsize=dsize, flags=cv2.INTER_LINEAR)
>>> print('result2 =\n{}'.format(ub.repr2(result2, precision=2)))
>>> # Ensure the results are the same (up to floating point errors)
>>> # NOTE: The errors seem to be significant for rotation / shear
>>> assert np.all(np.isclose(result1[0, 0].cpu().numpy(), result2, atol=1, rtol=1e-2))

Example

>>> # Test 3D warping with identity
>>> mat = torch.eye(4)
>>> input_dims = [2, 3, 3]
>>> output_dims = (2, 3, 3)
>>> input_shape = [1, 1] + input_dims
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> result = warp_tensor(inputs, mat, output_dims=output_dims)
>>> print('result =\n{}'.format(ub.repr2(result.cpu().numpy()[0, 0], precision=2)))
>>> assert torch.all(inputs == result)

Example

>>> # Test 3D warping with scaling
>>> mat = torch.FloatTensor([
>>>     [0.8,   0,   0, 0],
>>>     [  0, 1.0,   0, 0],
>>>     [  0,   0, 1.2, 0],
>>>     [  0,   0,   0, 1],
>>> ])
>>> input_dims = [2, 3, 3]
>>> output_dims = (2, 3, 3)
>>> input_shape = [1, 1] + input_dims
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> result = warp_tensor(inputs, mat, output_dims=output_dims, align_corners=0)
>>> print('result =\n{}'.format(ub.repr2(result.cpu().numpy()[0, 0], precision=2)))
result =
np.array([[[ 0.  ,  1.25,  1.  ],
           [ 3.  ,  4.25,  2.5 ],
           [ 6.  ,  7.25,  4.  ]],
          ...
          [[ 7.5 ,  8.75,  4.75],
           [10.5 , 11.75,  6.25],
           [13.5 , 14.75,  7.75]]], dtype=np.float32)

Example

>>> mat = torch.eye(3)
>>> input_dims = [5, 7]
>>> output_dims = (11, 7)
>>> for n_prefix_dims in [0, 1, 2, 3, 4, 5]:
>>>      input_shape = [2] * n_prefix_dims + input_dims
>>>      inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>>      result = warp_tensor(inputs, mat, output_dims=output_dims)
>>>      #print('result =\n{}'.format(ub.repr2(result.cpu().numpy(), precision=2)))
>>>      print(result.shape)

Example

>>> mat = torch.eye(4)
>>> input_dims = [5, 5, 5]
>>> output_dims = (6, 6, 6)
>>> for n_prefix_dims in [0, 1, 2, 3, 4, 5]:
>>>      input_shape = [2] * n_prefix_dims + input_dims
>>>      inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>>      result = warp_tensor(inputs, mat, output_dims=output_dims)
>>>      #print('result =\n{}'.format(ub.repr2(result.cpu().numpy(), precision=2)))
>>>      print(result.shape)
Ignore:
import xdev globals().update(xdev.get_func_kwargs(warp_tensor)) >>> import cv2 >>> inputs = torch.arange(9).view(1, 1, 3, 3).float() + 2 >>> input_dims = inputs.shape[2:] >>> #output_dims = (6, 6) >>> def fmt(a): >>> return ub.repr2(a.numpy(), precision=2) >>> s = 2.5 >>> output_dims = tuple(np.round((np.array(input_dims) * s)).astype(np.int).tolist()) >>> mat = torch.FloatTensor([[s, 0, 0], [0, s, 0], [0, 0, 1]]) >>> inv = mat.inverse() >>> warp_tensor(inputs, mat, output_dims) >>> print(‘## INPUTS’) >>> print(fmt(inputs)) >>> print(‘nalign_corners=True’) >>> print(‘—-‘) >>> print(‘## warp_tensor, align_corners=True’) >>> print(fmt(warp_tensor(inputs, inv, output_dims, isinv=True, align_corners=True))) >>> print(‘## interpolate, align_corners=True’) >>> print(fmt(F.interpolate(inputs, output_dims, mode=’bilinear’, align_corners=True))) >>> print(‘nalign_corners=False’) >>> print(‘—-‘) >>> print(‘## warp_tensor, align_corners=False, new_mode=False’) >>> print(fmt(warp_tensor(inputs, inv, output_dims, isinv=True, align_corners=False))) >>> print(‘## warp_tensor, align_corners=False, new_mode=True’) >>> print(fmt(warp_tensor(inputs, inv, output_dims, isinv=True, align_corners=False, new_mode=True))) >>> print(‘## interpolate, align_corners=False’) >>> print(fmt(F.interpolate(inputs, output_dims, mode=’bilinear’, align_corners=False))) >>> print(‘## interpolate (scale), align_corners=False’) >>> print(ub.repr2(F.interpolate(inputs, scale_factor=s, mode=’bilinear’, align_corners=False).numpy(), precision=2)) >>> cv2_M = mat.cpu().numpy()[0:2] >>> src = inputs[0, 0].cpu().numpy() >>> dsize = tuple(output_dims[::-1]) >>> print(‘nOpen CV warp Result’) >>> result2 = (cv2.warpAffine(src, cv2_M, dsize=dsize, flags=cv2.INTER_LINEAR)) >>> print(‘result2 =n{}’.format(ub.repr2(result2, precision=2)))
kwimage.util_warp.subpixel_align(dst, src, index, interp_axes=None)

Returns an aligned version of the source tensor and destination index.

Used as the backend to implement other subpixel functions like:
subpixel_accum, subpixel_maximum.
kwimage.util_warp.subpixel_set(dst, src, index, interp_axes=None)

Add the source values array into the destination array at a particular subpixel index.

Parameters:
  • dst (ArrayLike) – destination accumulation array
  • src (ArrayLike) – source array containing values to add
  • index (Tuple[slice]) – subpixel slice into dst that corresponds with src
  • interp_axes (tuple) – specify which axes should be spatially interpolated

Todo

  • [ ]: allow index to be a sequence indices

Example

>>> import kwimage
>>> dst = np.zeros(5) + .1
>>> src = np.ones(2)
>>> index = [slice(1.5, 3.5)]
>>> kwimage.util_warp.subpixel_set(dst, src, index)
>>> print(ub.repr2(dst, precision=2, with_dtype=0))
np.array([0.1, 0.5, 1. , 0.5, 0.1])
kwimage.util_warp.subpixel_accum(dst, src, index, interp_axes=None)

Add the source values array into the destination array at a particular subpixel index.

Parameters:
  • dst (ArrayLike) – destination accumulation array
  • src (ArrayLike) – source array containing values to add
  • index (Tuple[slice]) – subpixel slice into dst that corresponds with src
  • interp_axes (tuple) – specify which axes should be spatially interpolated

Notes

Inputs:
+—+—+—+—+—+ dst.shape = (5,)
+—+—+ src.shape = (2,) |=======| index = 1.5:3.5

Subpixel shift the source by -0.5. When the index is non-integral, pad the aligned src with an extra value to ensure all dst pixels that would be influenced by the smaller subpixel shape are influenced by the aligned src. Note that we are not scaling.

+—+—+—+ aligned_src.shape = (3,) |===========| aligned_index = 1:4

Example

>>> dst = np.zeros(5)
>>> src = np.ones(2)
>>> index = [slice(1.5, 3.5)]
>>> subpixel_accum(dst, src, index)
>>> print(ub.repr2(dst, precision=2, with_dtype=0))
np.array([0. , 0.5, 1. , 0.5, 0. ])

Example

>>> dst = np.zeros((6, 6))
>>> src = np.ones((3, 3))
>>> index = (slice(1.5, 4.5), slice(1, 4))
>>> subpixel_accum(dst, src, index)
>>> print(ub.repr2(dst, precision=2, with_dtype=0))
np.array([[0. , 0. , 0. , 0. , 0. , 0. ],
          [0. , 0.5, 0.5, 0.5, 0. , 0. ],
          [0. , 1. , 1. , 1. , 0. , 0. ],
          [0. , 1. , 1. , 1. , 0. , 0. ],
          [0. , 0.5, 0.5, 0.5, 0. , 0. ],
          [0. , 0. , 0. , 0. , 0. , 0. ]])
>>> dst = torch.zeros((1, 3, 6, 6))
>>> src = torch.ones((1, 3, 3, 3))
>>> index = (slice(None), slice(None), slice(1.5, 4.5), slice(1.25, 4.25))
>>> subpixel_accum(dst, src, index)
>>> print(ub.repr2(dst.numpy()[0, 0], precision=2, with_dtype=0))
np.array([[0.  , 0.  , 0.  , 0.  , 0.  , 0.  ],
          [0.  , 0.38, 0.5 , 0.5 , 0.12, 0.  ],
          [0.  , 0.75, 1.  , 1.  , 0.25, 0.  ],
          [0.  , 0.75, 1.  , 1.  , 0.25, 0.  ],
          [0.  , 0.38, 0.5 , 0.5 , 0.12, 0.  ],
          [0.  , 0.  , 0.  , 0.  , 0.  , 0.  ]])
Doctest:
>>> # TODO: move to a unit test file
>>> subpixel_accum(np.zeros(5), np.ones(2), [slice(1.5, 3.5)]).tolist()
[0.0, 0.5, 1.0, 0.5, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(2), [slice(0, 2)]).tolist()
[1.0, 1.0, 0.0, 0.0, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(.5, 3.5)]).tolist()
[0.5, 1.0, 1.0, 0.5, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(-1, 2)]).tolist()
[1.0, 1.0, 0.0, 0.0, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(-1.5, 1.5)]).tolist()
[1.0, 0.5, 0.0, 0.0, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(10, 13)]).tolist()
[0.0, 0.0, 0.0, 0.0, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(3.25, 6.25)]).tolist()
[0.0, 0.0, 0.0, 0.75, 1.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(4.9, 7.9)]).tolist()
[0.0, 0.0, 0.0, 0.0, 0.099...]
>>> subpixel_accum(np.zeros(5), np.ones(9), [slice(-1.5, 7.5)]).tolist()
[1.0, 1.0, 1.0, 1.0, 1.0]
>>> subpixel_accum(np.zeros(5), np.ones(9), [slice(2.625, 11.625)]).tolist()
[0.0, 0.0, 0.375, 1.0, 1.0]
>>> subpixel_accum(np.zeros(5), 1, [slice(2.625, 11.625)]).tolist()
[0.0, 0.0, 0.375, 1.0, 1.0]
kwimage.util_warp.subpixel_maximum(dst, src, index, interp_axes=None)

Take the max of the source values array into and the destination array at a particular subpixel index. Modifies the destination array.

Parameters:
  • dst (ArrayLike) – destination array to index into
  • src (ArrayLike) – source array that agrees with the index
  • index (Tuple[slice]) – subpixel slice into dst that corresponds with src
  • interp_axes (tuple) – specify which axes should be spatially interpolated

Example

>>> dst = np.array([0, 1.0, 1.0, 1.0, 0])
>>> src = np.array([2.0, 2.0])
>>> index = [slice(1.6, 3.6)]
>>> subpixel_maximum(dst, src, index)
>>> print(ub.repr2(dst, precision=2, with_dtype=0))
np.array([0. , 1. , 2. , 1.2, 0. ])

Example

>>> dst = torch.zeros((1, 3, 5, 5)) + .5
>>> src = torch.ones((1, 3, 3, 3))
>>> index = (slice(None), slice(None), slice(1.4, 4.4), slice(1.25, 4.25))
>>> subpixel_maximum(dst, src, index)
>>> print(ub.repr2(dst.numpy()[0, 0], precision=2, with_dtype=0))
np.array([[0.5 , 0.5 , 0.5 , 0.5 , 0.5 ],
          [0.5 , 0.5 , 0.6 , 0.6 , 0.5 ],
          [0.5 , 0.75, 1.  , 1.  , 0.5 ],
          [0.5 , 0.75, 1.  , 1.  , 0.5 ],
          [0.5 , 0.5 , 0.5 , 0.5 , 0.5 ]])
kwimage.util_warp.subpixel_minimum(dst, src, index, interp_axes=None)

Take the min of the source values array into and the destination array at a particular subpixel index. Modifies the destination array.

Parameters:
  • dst (ArrayLike) – destination array to index into
  • src (ArrayLike) – source array that agrees with the index
  • index (Tuple[slice]) – subpixel slice into dst that corresponds with src
  • interp_axes (tuple) – specify which axes should be spatially interpolated

Example

>>> dst = np.array([0, 1.0, 1.0, 1.0, 0])
>>> src = np.array([2.0, 2.0])
>>> index = [slice(1.6, 3.6)]
>>> subpixel_minimum(dst, src, index)
>>> print(ub.repr2(dst, precision=2, with_dtype=0))
np.array([0. , 0.8, 1. , 1. , 0. ])

Example

>>> dst = torch.zeros((1, 3, 5, 5)) + .5
>>> src = torch.ones((1, 3, 3, 3))
>>> index = (slice(None), slice(None), slice(1.4, 4.4), slice(1.25, 4.25))
>>> subpixel_minimum(dst, src, index)
>>> print(ub.repr2(dst.numpy()[0, 0], precision=2, with_dtype=0))
np.array([[0.5 , 0.5 , 0.5 , 0.5 , 0.5 ],
          [0.5 , 0.45, 0.5 , 0.5 , 0.15],
          [0.5 , 0.5 , 0.5 , 0.5 , 0.25],
          [0.5 , 0.5 , 0.5 , 0.5 , 0.25],
          [0.5 , 0.3 , 0.4 , 0.4 , 0.1 ]])
kwimage.util_warp.subpixel_slice(inputs, index)

Take a subpixel slice from a larger image. The returned output is left-aligned with the requested slice.

Parameters:
  • inputs (ArrayLike) – data
  • index (Tuple[slice]) – a slice to subpixel accuracy

Example

>>> inputs = np.arange(5 * 5 * 3).reshape(5, 5, 3)
>>> index = [slice(0, 3), slice(0, 3)]
>>> outputs = subpixel_slice(inputs, index)
>>> index = [slice(0.5, 3.5), slice(-0.5, 2.5)]
>>> outputs = subpixel_slice(inputs, index)
>>> inputs = np.arange(5 * 5).reshape(1, 5, 5).astype(np.float)
>>> index = [slice(None), slice(3, 6), slice(3, 6)]
>>> outputs = subpixel_slice(inputs, index)
>>> print(outputs)
[[[18. 19.  0.]
  [23. 24.  0.]
  [ 0.  0.  0.]]]
>>> index = [slice(None), slice(3.5, 6.5), slice(2.5, 5.5)]
>>> outputs = subpixel_slice(inputs, index)
>>> print(outputs)
[[[20.   21.   10.75]
  [11.25 11.75  6.  ]
  [ 0.    0.    0.  ]]]
kwimage.util_warp.subpixel_translate(inputs, shift, interp_axes=None, output_shape=None)

Translates an image by a subpixel shift value using bilinear interpolation

Parameters:
  • inputs (ArrayLike) – data to translate
  • shift (Sequence) – amount to translate each dimension specified by interp_axes. Note: if inputs contains more than one “image” then all “images” are translated by the same amount. This function contains no mechanism for translating each image differently. Note that by default this is a y,x shift for 2 dimensions.
  • interp_axes (Sequence, default=None) – axes to perform interpolation on, if not specified the final n axes are interpolated, where n=len(shift)
  • output_shape (tuple, default=None) – if specified the output is returned with this shape, otherwise

Notes

This function powers most other functions in this file. Speedups here can go a long way.

Example

>>> inputs = np.arange(5) + 1
>>> print(inputs.tolist())
[1, 2, 3, 4, 5]
>>> outputs = subpixel_translate(inputs, 1.5)
>>> print(outputs.tolist())
[0.0, 0.5, 1.5, 2.5, 3.5]

Example

>>> inputs = torch.arange(9).view(1, 1, 3, 3).float()
>>> print(inputs.long())
tensor([[[[0, 1, 2],
          [3, 4, 5],
          [6, 7, 8]]]])
>>> outputs = subpixel_translate(inputs, (-.4, .5), output_shape=(1, 1, 2, 5))
>>> print(outputs)
tensor([[[[0.6000, 1.7000, 2.7000, 1.6000, 0.0000],
          [2.1000, 4.7000, 5.7000, 3.1000, 0.0000]]]])
Ignore:
>>> inputs = np.arange(5)
>>> shift = -.6
>>> interp_axes = None
>>> subpixel_translate(inputs, -.6)
>>> subpixel_translate(inputs[None, None, None, :], -.6)
>>> inputs = np.arange(25).reshape(5, 5)
>>> shift = (-1.6, 2.3)
>>> interp_axes = (0, 1)
>>> subpixel_translate(inputs, shift, interp_axes, output_shape=(9, 9))
>>> subpixel_translate(inputs, shift, interp_axes, output_shape=(3, 4))
kwimage.util_warp._padded_slice(data, in_slice, ndim=None, pad_slice=None, pad_mode='constant', **padkw)

Allows slices with out-of-bound coordinates. Any out of bounds coordinate will be sampled via padding.

Note

Negative slices have a different meaning here then they usually do. Normally, they indicate a wrap-around or a reversed stride, but here they index into out-of-bounds space (which depends on the pad mode). For example a slice of -2:1 literally samples two pixels to the left of the data and one pixel from the data, so you get two padded values and one data value.

Parameters:
  • data (Sliceable[T]) – data to slice into. Any channels must be the last dimension.
  • in_slice (Tuple[slice, …]) – slice for each dimensions
  • ndim (int) – number of spatial dimensions
  • pad_slice (List[int|Tuple]) – additional padding of the slice
Returns:

data_sliced: subregion of the input data (possibly with padding,

depending on if the original slice went out of bounds)

st_dims : a list indicating the low and high space-time coordinate

values of the returned data slice.

Return type:

Tuple[Sliceable, List]

Example

>>> data = np.arange(5)
>>> in_slice = [slice(-2, 7)]
>>> data_sliced, st_dims = _padded_slice(data, in_slice)
>>> print(ub.repr2(data_sliced, with_dtype=False))
>>> print(st_dims)
np.array([0, 0, 0, 1, 2, 3, 4, 0, 0])
[(-2, 7)]
>>> data_sliced, st_dims = _padded_slice(data, in_slice, pad_slice=(3, 3))
>>> print(ub.repr2(data_sliced, with_dtype=False))
>>> print(st_dims)
np.array([0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 0, 0, 0, 0, 0])
[(-5, 10)]
>>> data_sliced, st_dims = _padded_slice(data, slice(3, 4), pad_slice=[(1, 0)])
>>> print(ub.repr2(data_sliced, with_dtype=False))
>>> print(st_dims)
np.array([2, 3])
[(2, 4)]
kwimage.util_warp._ensure_arraylike(data, n=None)
kwimage.util_warp._rectify_slice(data_dims, low_dims, high_dims, pad_slice=None)

Given image dimensions, bounding box dimensions, and a padding get the corresponding slice from the image and any extra padding needed to achieve the requested window size.

Parameters:
  • data_dims (tuple) – n-dimension data sizes (e.g. 2d height, width)
  • low_dims (tuple) – bounding box low values (e.g. 2d ymin, xmin)
  • high_dims (tuple) – bounding box high values (e.g. 2d ymax, xmax)
  • pad_slice (List[int|Tuple]) – pad applied to (left and right) / (both) sides of each slice dim
Returns:

data_slice - low and high values of a fancy slice corresponding to

the image with shape data_dims. This slice may not correspond to the full window size if the requested bounding box goes out of bounds.

extra_padding - extra padding needed after slicing to achieve

the requested window size.

Return type:

Tuple

Example

>>> # Case where 2D-bbox is inside the data dims on left edge
>>> # Comprehensive 1D-cases are in the unit-test file
>>> data_dims  = [300, 300]
>>> low_dims   = [0, 0]
>>> high_dims  = [10, 10]
>>> pad_slice  = [(10, 10), (5, 5)]
>>> a, b = _rectify_slice(data_dims, low_dims, high_dims, pad_slice)
>>> print('data_slice = {!r}'.format(a))
>>> print('extra_padding = {!r}'.format(b))
data_slice = [(0, 20), (0, 15)]
extra_padding = [(10, 0), (5, 0)]
kwimage.util_warp._warp_tensor_cv2(inputs, mat, output_dims, mode='linear', ishomog=None)

implementation with cv2.warpAffine for speed / correctness comparison

On GPU: torch is faster in both modes On CPU: torch is faster for homog, but cv2 is faster for affine

Benchmark:
>>> from kwimage.util.util_warp import *
>>> from kwimage.util.util_warp import _warp_tensor_cv2
>>> from kwimage.util.util_warp import warp_tensor
>>> import numpy as np
>>> ti = ub.Timerit(10, bestof=3, verbose=2, unit='ms')
>>> mode = 'linear'
>>> rng = np.random.RandomState(0)
>>> inputs = torch.Tensor(rng.rand(16, 10, 32, 32)).to('cpu')
>>> mat = torch.FloatTensor([[2.5, 0, 10.5], [0, 3, 0], [0, 0, 1]])
>>> mat[2, 0] = .009
>>> mat[2, 2] = 2
>>> output_dims = (64, 64)
>>> results = ub.odict()
>>> # -------------
>>> for timer in ti.reset('warp_tensor(torch)'):
>>>     with timer:
>>>         outputs = warp_tensor(inputs, mat, output_dims, mode=mode)
>>>         torch.cuda.synchronize()
>>> results[ti.label] = outputs
>>> # -------------
>>> inputs = inputs.cpu().numpy()
>>> mat = mat.cpu().numpy()
>>> for timer in ti.reset('warp_tensor(cv2)'):
>>>     with timer:
>>>         outputs = _warp_tensor_cv2(inputs, mat, output_dims, mode=mode)
>>> results[ti.label] = outputs
>>> import itertools as it
>>> for k1, k2 in it.combinations(results, 2):
>>>     a = kwarray.ArrayAPI.numpy(results[k1])
>>>     b = kwarray.ArrayAPI.numpy(results[k2])
>>>     diff = np.abs(a - b)
>>>     diff_stats = kwarray.stats_dict(diff, n_extreme=1, extreme=1)
>>>     print('{} - {}: {}'.format(k1, k2, ub.repr2(diff_stats, nl=0, precision=4)))
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autompl()
>>> kwplot.imshow(results['warp_tensor(torch)'][0, 0], fnum=1, pnum=(1, 2, 1), title='torch')
>>> kwplot.imshow(results['warp_tensor(cv2)'][0, 0], fnum=1, pnum=(1, 2, 2), title='cv2')
kwimage.util_warp.warp_points(matrix, pts, homog_mode='divide')

Warp ND points / coordinates using a transformation matrix.

Homogoenous coordinates are added on the fly if needed. Works with both numpy and torch.

Parameters:
  • matrix (ArrayLike) – [D1 x D2] transformation matrix. if using homogenous coordinates D2=D + 1, otherwise D2=D. if using homogenous coordinates and the matrix represents an Affine transformation, then either D1=D or D1=D2, i.e. the last row of zeros and a one is optional.
  • pts (ArrayLike) – [N1 x … x D] points (usually x, y). If points are already in homogenous space, then the output will be returned in homogenous space. D is the dimensionality of the points. The leading axis may take any shape, but usually, shape will be [N x D] where N is the number of points.
  • homog_mode (str, default=’divide’) – what to do for homogenous coordinates. Can either divide, keep, or drop.
Retrns:
new_pts (ArrayLike): the points after being transformed by the matrix

Example

>>> from kwimage.util_warp import *  # NOQA
>>> # --- with numpy
>>> rng = np.random.RandomState(0)
>>> pts = rng.rand(10, 2)
>>> matrix = rng.rand(2, 2)
>>> warp_points(matrix, pts)
>>> # --- with torch
>>> pts = torch.Tensor(pts)
>>> matrix = torch.Tensor(matrix)
>>> warp_points(matrix, pts)

Example

>>> from kwimage.util_warp import *  # NOQA
>>> # --- with numpy
>>> pts = np.ones((10, 2))
>>> matrix = np.diag([2, 3, 1])
>>> ra = warp_points(matrix, pts)
>>> rb = warp_points(torch.Tensor(matrix), torch.Tensor(pts))
>>> assert np.allclose(ra, rb.numpy())

Example

>>> from kwimage.util_warp import *  # NOQA
>>> # test different cases
>>> rng = np.random.RandomState(0)
>>> # Test 3x3 style projective matrices
>>> pts = rng.rand(1000, 2)
>>> matrix = rng.rand(3, 3)
>>> ra33 = warp_points(matrix, pts)
>>> rb33 = warp_points(torch.Tensor(matrix), torch.Tensor(pts))
>>> assert np.allclose(ra33, rb33.numpy())
>>> # Test opencv style affine matrices
>>> pts = rng.rand(10, 2)
>>> matrix = rng.rand(2, 3)
>>> ra23 = warp_points(matrix, pts)
>>> rb23 = warp_points(torch.Tensor(matrix), torch.Tensor(pts))
>>> assert np.allclose(ra33, rb33.numpy())
kwimage.util_warp.remove_homog(pts, mode='divide')

Remove homogenous coordinate to a point array.

This is a convinience function, it is not particularly efficient.

SeeAlso:
cv2.convertPointsFromHomogeneous

Example

>>> homog_pts = np.random.rand(10, 3)
>>> remove_homog(homog_pts, 'divide')
>>> remove_homog(homog_pts, 'drop')
kwimage.util_warp.add_homog(pts)

Add a homogenous coordinate to a point array

This is a convinience function, it is not particularly efficient.

SeeAlso:
cv2.convertPointsToHomogeneous

Example

>>> pts = np.random.rand(10, 2)
>>> add_homog(pts)
Benchmark:
>>> import timerit
>>> ti = timerit.Timerit(1000, bestof=10, verbose=2)
>>> pts = np.random.rand(1000, 2)
>>> for timer in ti.reset('kwimage'):
>>>     with timer:
>>>         kwimage.add_homog(pts)
>>> for timer in ti.reset('cv2'):
>>>     with timer:
>>>         cv2.convertPointsToHomogeneous(pts)
>>> # cv2 is 4x faster, but has more restrictive inputs
kwimage.util_warp.subpixel_getvalue(img, pts, coord_axes=None, interp='bilinear', bordermode='edge')

Get values at subpixel locations

Parameters:
  • img (ArrayLike) – image to sample from
  • pts (ArrayLike) – subpixel rc-coordinates to sample
  • coord_axes (Sequence, default=None) – axes to perform interpolation on, if not specified the first d axes are interpolated, where d=pts.shape[-1]. IE: this indicates which axes each coordinate dimension corresponds to.
  • interp (str) – interpolation mode
  • bordermode (str) – how locations outside the image are handled

Example

>>> from kwimage.util_warp import *  # NOQA
>>> img = np.arange(3 * 3).reshape(3, 3)
>>> pts = np.array([[1, 1], [1.5, 1.5], [1.9, 1.1]])
>>> subpixel_getvalue(img, pts)
array([4. , 6. , 6.8])
>>> subpixel_getvalue(img, pts, coord_axes=(1, 0))
array([4. , 6. , 5.2])
>>> img = torch.Tensor(img)
>>> pts = torch.Tensor(pts)
>>> subpixel_getvalue(img, pts)
tensor([4.0000, 6.0000, 6.8000])
>>> subpixel_getvalue(img.numpy(), pts.numpy(), interp='nearest')
array([4., 8., 7.], dtype=float32)
>>> subpixel_getvalue(img.numpy(), pts.numpy(), interp='nearest', coord_axes=[1, 0])
array([4., 8., 5.], dtype=float32)
>>> subpixel_getvalue(img, pts, interp='nearest')
tensor([4., 8., 7.])

References

stackoverflow.com/uestions/12729228/simple-binlin-interp-images-numpy

SeeAlso:
cv2.getRectSubPix(image, patchSize, center[, patch[, patchType]])
kwimage.util_warp.subpixel_setvalue(img, pts, value, coord_axes=None, interp='bilinear', bordermode='edge')

Set values at subpixel locations

Parameters:
  • img (ArrayLike) – image to set values in
  • pts (ArrayLike) – subpixel rc-coordinates to set
  • value (ArrayLike) – value to place in the image
  • coord_axes (Sequence, default=None) – axes to perform interpolation on, if not specified the first d axes are interpolated, where d=pts.shape[-1]. IE: this indicates which axes each coordinate dimension corresponds to.
  • interp (str) – interpolation mode
  • bordermode (str) – how locations outside the image are handled

Example

>>> from kwimage.util_warp import *  # NOQA
>>> img = np.arange(3 * 3).reshape(3, 3).astype(np.float)
>>> pts = np.array([[1, 1], [1.5, 1.5], [1.9, 1.1]])
>>> interp = 'bilinear'
>>> value = 0
>>> print('img = {!r}'.format(img))
>>> pts = np.array([[1.5, 1.5]])
>>> img2 = subpixel_setvalue(img.copy(), pts, value)
>>> print('img2 = {!r}'.format(img2))
>>> pts = np.array([[1.0, 1.0]])
>>> img2 = subpixel_setvalue(img.copy(), pts, value)
>>> print('img2 = {!r}'.format(img2))
>>> pts = np.array([[1.1, 1.9]])
>>> img2 = subpixel_setvalue(img.copy(), pts, value)
>>> print('img2 = {!r}'.format(img2))
>>> img2 = subpixel_setvalue(img.copy(), pts, value, coord_axes=[1, 0])
>>> print('img2 = {!r}'.format(img2))
kwimage.util_warp._bilinear_coords(ptsT, impl, img, coord_axes)