kwimage.util_warp module

Todo

• [ ] Replace internal padded slice with kwarray.padded_slice

kwimage.util_warp._coordinate_grid(dims, align_corners=False)

Creates a homogenous coordinate system.

Parameters:

• dims (Tuple[int, …]) – height / width or depth / height / width

• align_corners (bool) – returns a grid where the left and right corners assigned to the extreme values and intermediate values are interpolated.

Returns:

with shape=(3, *DIMS)

Return type:

Tensor

References

[InvWarp]

https://github.com/ClementPinard/SfmLearner-Pytorch/blob/master/inverse_warp.py

Example

>>> # xdoctest: +IGNORE_WHITESPACE
>>> # xdoctest: +REQUIRES(module:torch)
>>> _coordinate_grid((2, 2))
tensor([[[0., 1.],
         [0., 1.]],
        [[0., 0.],
         [1., 1.]],
        [[1., 1.],
         [1., 1.]]])
>>> _coordinate_grid((2, 2, 2))
>>> _coordinate_grid((2, 2), align_corners=True)
tensor([[[0., 2.],
         [0., 2.]],
        [[0., 0.],
         [2., 2.]],
        [[1., 1.],
         [1., 1.]]])

kwimage.util_warp.warp_tensor(inputs, mat, output_dims, mode='bilinear', padding_mode='zeros', isinv=False, ishomog=None, align_corners=False, new_mode=False)

A pytorch implementation of warp affine that works similarly to cv2.warpAffine() and cv2.warpPerspective().

It is possible to use 3x3 transforms to warp 2D image data. It is also possible to use 4x4 transforms to warp 3D volumetric data.

Parameters:

• inputs (Tensor) – tensor to warp. Up to 3 (determined by output_dims) of the trailing space-time dimensions are warped. Best practice is to use inputs with the shape in [B, C, *DIMS].

• mat (Tensor) – either a 3x3 / 4x4 single transformation matrix to apply to all inputs or Bx3x3 or Bx4x4 tensor that specifies a transformation matrix for each batch item.

• output_dims (Tuple[int, …]) – The output space-time dimensions. This can either be in the form (W,), (H, W), or (D, H, W).

• mode (str) – Can be bilinear or nearest. See torch.nn.functional.grid_sample

• padding_mode (str) – Can be zeros, border, or reflection. See torch.nn.functional.grid_sample.

• isinv (bool) – Set to true if mat is the inverse transform

• ishomog (bool) – Set to True if the matrix is non-affine

• align_corners (bool) – Note the default of False does not work correctly with grid_sample in torch <= 1.2, but using align_corners=True isnt typically what you want either. We will be stuck with buggy functionality until torch 1.3 is released.

  However, using align_corners=0 does seem to reasonably correspond with opencv behavior.

Returns:

warped tensor

Return type:

Tensor

Note

Also, it may be possible to speed up the code with F.affine_grid

KNOWN ISSUE: There appears to some difference with cv2.warpAffine when

rotation or shear are non-zero. I’m not sure what the cause is. It may just be floating point issues, but Im’ not sure.

See issues in [TorchAffineTransform] and [TorchIssue15386].

Todo

• [ ] FIXME: see example in Mask.scale where this algo breaks when the matrix is 2x3

• [ ] Make this algo work when matrix ix 2x2

References

[TorchAffineTransform]

https://discuss.pytorch.org/t/affine-transformation-matrix-paramters-conversion/19522

[TorchIssue15386]

https://github.com/pytorch/pytorch/issues/15386

Example

>>> # Create a relatively simple affine matrix
>>> # xdoctest: +REQUIRES(module:torch)
>>> import skimage
>>> import torch
>>> mat = torch.FloatTensor(skimage.transform.AffineTransform(
>>>     translation=[1, -1], scale=[.532, 2],
>>>     rotation=0, shear=0,
>>> ).params)
>>> # Create inputs and an output dimension
>>> input_shape = [1, 1, 4, 5]
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> output_dims = (11, 7)
>>> # Warp with our code
>>> result1 = warp_tensor(inputs, mat, output_dims=output_dims, align_corners=0)
>>> print('result1 =\n{}'.format(ub.urepr(result1.cpu().numpy()[0, 0], precision=2)))
>>> # Warp with opencv
>>> import cv2
>>> cv2_M = mat.cpu().numpy()[0:2]
>>> src = inputs[0, 0].cpu().numpy()
>>> dsize = tuple(output_dims[::-1])
>>> result2 = cv2.warpAffine(src, cv2_M, dsize=dsize, flags=cv2.INTER_LINEAR)
>>> print('result2 =\n{}'.format(ub.urepr(result2, precision=2)))
>>> # Ensure the results are the same (up to floating point errors)
>>> assert np.all(np.isclose(result1[0, 0].cpu().numpy(), result2, atol=1e-2, rtol=1e-2))

Example

>>> # Create a relatively simple affine matrix
>>> # xdoctest: +REQUIRES(module:torch)
>>> import skimage
>>> import torch
>>> mat = torch.FloatTensor(skimage.transform.AffineTransform(
>>>     rotation=0.01, shear=0.1).params)
>>> # Create inputs and an output dimension
>>> input_shape = [1, 1, 4, 5]
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> output_dims = (11, 7)
>>> # Warp with our code
>>> result1 = warp_tensor(inputs, mat, output_dims=output_dims)
>>> print('result1 =\n{}'.format(ub.urepr(result1.cpu().numpy()[0, 0], precision=2, supress_small=True)))
>>> print('result1.shape = {}'.format(result1.shape))
>>> # Warp with opencv
>>> import cv2
>>> cv2_M = mat.cpu().numpy()[0:2]
>>> src = inputs[0, 0].cpu().numpy()
>>> dsize = tuple(output_dims[::-1])
>>> result2 = cv2.warpAffine(src, cv2_M, dsize=dsize, flags=cv2.INTER_LINEAR)
>>> print('result2 =\n{}'.format(ub.urepr(result2, precision=2)))
>>> print('result2.shape = {}'.format(result2.shape))
>>> # Ensure the results are the same (up to floating point errors)
>>> # NOTE: The floating point errors seem to be significant for rotation / shear
>>> assert np.all(np.isclose(result1[0, 0].cpu().numpy(), result2, atol=1, rtol=1e-2))

Example

>>> # Create a random affine matrix
>>> # xdoctest: +REQUIRES(module:torch)
>>> import skimage
>>> import torch
>>> rng = np.random.RandomState(0)
>>> mat = torch.FloatTensor(skimage.transform.AffineTransform(
>>>     translation=rng.randn(2), scale=1 + rng.randn(2),
>>>     rotation=rng.randn() / 10., shear=rng.randn() / 10.,
>>> ).params)
>>> # Create inputs and an output dimension
>>> input_shape = [1, 1, 5, 7]
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> output_dims = (3, 11)
>>> # Warp with our code
>>> result1 = warp_tensor(inputs, mat, output_dims=output_dims, align_corners=0)
>>> print('result1 =\n{}'.format(ub.urepr(result1.cpu().numpy()[0, 0], precision=2)))
>>> # Warp with opencv
>>> import cv2
>>> cv2_M = mat.cpu().numpy()[0:2]
>>> src = inputs[0, 0].cpu().numpy()
>>> dsize = tuple(output_dims[::-1])
>>> result2 = cv2.warpAffine(src, cv2_M, dsize=dsize, flags=cv2.INTER_LINEAR)
>>> print('result2 =\n{}'.format(ub.urepr(result2, precision=2)))
>>> # Ensure the results are the same (up to floating point errors)
>>> # NOTE: The errors seem to be significant for rotation / shear
>>> assert np.all(np.isclose(result1[0, 0].cpu().numpy(), result2, atol=1, rtol=1e-2))

Example

>>> # Test 3D warping with identity
>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> mat = torch.eye(4)
>>> input_dims = [2, 3, 3]
>>> output_dims = (2, 3, 3)
>>> input_shape = [1, 1] + input_dims
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> result = warp_tensor(inputs, mat, output_dims=output_dims)
>>> print('result =\n{}'.format(ub.urepr(result.cpu().numpy()[0, 0], precision=2)))
>>> assert torch.all(inputs == result)

Example

>>> # Test 3D warping with scaling
>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> mat = torch.FloatTensor([
>>>     [0.8,   0,   0, 0],
>>>     [  0, 1.0,   0, 0],
>>>     [  0,   0, 1.2, 0],
>>>     [  0,   0,   0, 1],
>>> ])
>>> input_dims = [2, 3, 3]
>>> output_dims = (2, 3, 3)
>>> input_shape = [1, 1] + input_dims
>>> inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>> result = warp_tensor(inputs, mat, output_dims=output_dims, align_corners=0)
>>> print('result =\n{}'.format(ub.urepr(result.cpu().numpy()[0, 0], precision=2)))
result =
np.array([[[ 0.  ,  1.25,  1.  ],
           [ 3.  ,  4.25,  2.5 ],
           [ 6.  ,  7.25,  4.  ]],
          ...
          [[ 7.5 ,  8.75,  4.75],
           [10.5 , 11.75,  6.25],
           [13.5 , 14.75,  7.75]]], dtype=np.float32)

Example

>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> mat = torch.eye(3)
>>> input_dims = [5, 7]
>>> output_dims = (11, 7)
>>> for n_prefix_dims in [0, 1, 2, 3, 4, 5]:
>>>      input_shape = [2] * n_prefix_dims + input_dims
>>>      inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>>      result = warp_tensor(inputs, mat, output_dims=output_dims)
>>>      #print('result =\n{}'.format(ub.urepr(result.cpu().numpy(), precision=2)))
>>>      print(result.shape)

Example

>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> mat = torch.eye(4)
>>> input_dims = [5, 5, 5]
>>> output_dims = (6, 6, 6)
>>> for n_prefix_dims in [0, 1, 2, 3, 4, 5]:
>>>      input_shape = [2] * n_prefix_dims + input_dims
>>>      inputs = torch.arange(int(np.prod(input_shape))).reshape(*input_shape).float()
>>>      result = warp_tensor(inputs, mat, output_dims=output_dims)
>>>      #print('result =\n{}'.format(ub.urepr(result.cpu().numpy(), precision=2)))
>>>      print(result.shape)

kwimage.util_warp.subpixel_align(dst, src, index, interp_axes=None)

Returns an aligned version of the source tensor and destination index.

Used as the backend to implement other subpixel functions like:

subpixel_accum, subpixel_maximum.

kwimage.util_warp.subpixel_set(dst, src, index, interp_axes=None)

Add the source values array into the destination array at a particular subpixel index.

Parameters:

• dst (ArrayLike) – destination accumulation array

• src (ArrayLike) – source array containing values to add

• index (Tuple[slice]) – subpixel slice into dst that corresponds with src

• interp_axes (tuple) – specify which axes should be spatially interpolated

Todo

• [ ]: allow index to be a sequence indices

Example

>>> import kwimage
>>> dst = np.zeros(5) + .1
>>> src = np.ones(2)
>>> index = [slice(1.5, 3.5)]
>>> kwimage.util_warp.subpixel_set(dst, src, index)
>>> print(ub.urepr(dst, precision=2, with_dtype=0))
np.array([0.1, 0.5, 1. , 0.5, 0.1])

kwimage.util_warp.subpixel_accum(dst, src, index, interp_axes=None)

Add the source values array into the destination array at a particular subpixel index.

Parameters:

• dst (ArrayLike) – destination accumulation array

• src (ArrayLike) – source array containing values to add

• index (Tuple[slice]) – subpixel slice into dst that corresponds with src

• interp_axes (tuple) – specify which axes should be spatially interpolated

TextArt

Inputs:
    +---+---+---+---+---+  dst.shape = (5,)
          +---+---+        src.shape = (2,)
          |=======|        index = 1.5:3.5

Subpixel shift the source by -0.5.
When the index is non-integral, pad the aligned src with an extra value
to ensure all dst pixels that would be influenced by the smaller
subpixel shape are influenced by the aligned src. Note that we are not
scaling.

        +---+---+---+      aligned_src.shape = (3,)
        |===========|      aligned_index = 1:4

Example

>>> dst = np.zeros(5)
>>> src = np.ones(2)
>>> index = [slice(1.5, 3.5)]
>>> subpixel_accum(dst, src, index)
>>> print(ub.urepr(dst, precision=2, with_dtype=0))
np.array([0. , 0.5, 1. , 0.5, 0. ])

Example

>>> dst = np.zeros((6, 6))
>>> src = np.ones((3, 3))
>>> index = (slice(1.5, 4.5), slice(1, 4))
>>> subpixel_accum(dst, src, index)
>>> print(ub.urepr(dst, precision=2, with_dtype=0))
np.array([[0. , 0. , 0. , 0. , 0. , 0. ],
          [0. , 0.5, 0.5, 0.5, 0. , 0. ],
          [0. , 1. , 1. , 1. , 0. , 0. ],
          [0. , 1. , 1. , 1. , 0. , 0. ],
          [0. , 0.5, 0.5, 0.5, 0. , 0. ],
          [0. , 0. , 0. , 0. , 0. , 0. ]])
>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> dst = torch.zeros((1, 3, 6, 6))
>>> src = torch.ones((1, 3, 3, 3))
>>> index = (slice(None), slice(None), slice(1.5, 4.5), slice(1.25, 4.25))
>>> subpixel_accum(dst, src, index)
>>> print(ub.urepr(dst.numpy()[0, 0], precision=2, with_dtype=0))
np.array([[0.  , 0.  , 0.  , 0.  , 0.  , 0.  ],
          [0.  , 0.38, 0.5 , 0.5 , 0.12, 0.  ],
          [0.  , 0.75, 1.  , 1.  , 0.25, 0.  ],
          [0.  , 0.75, 1.  , 1.  , 0.25, 0.  ],
          [0.  , 0.38, 0.5 , 0.5 , 0.12, 0.  ],
          [0.  , 0.  , 0.  , 0.  , 0.  , 0.  ]])

Doctest

>>> # TODO: move to a unit test file
>>> subpixel_accum(np.zeros(5), np.ones(2), [slice(1.5, 3.5)]).tolist()
[0.0, 0.5, 1.0, 0.5, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(2), [slice(0, 2)]).tolist()
[1.0, 1.0, 0.0, 0.0, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(.5, 3.5)]).tolist()
[0.5, 1.0, 1.0, 0.5, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(-1, 2)]).tolist()
[1.0, 1.0, 0.0, 0.0, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(-1.5, 1.5)]).tolist()
[1.0, 0.5, 0.0, 0.0, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(10, 13)]).tolist()
[0.0, 0.0, 0.0, 0.0, 0.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(3.25, 6.25)]).tolist()
[0.0, 0.0, 0.0, 0.75, 1.0]
>>> subpixel_accum(np.zeros(5), np.ones(3), [slice(4.9, 7.9)]).tolist()
[0.0, 0.0, 0.0, 0.0, 0.099...]
>>> subpixel_accum(np.zeros(5), np.ones(9), [slice(-1.5, 7.5)]).tolist()
[1.0, 1.0, 1.0, 1.0, 1.0]
>>> subpixel_accum(np.zeros(5), np.ones(9), [slice(2.625, 11.625)]).tolist()
[0.0, 0.0, 0.375, 1.0, 1.0]
>>> subpixel_accum(np.zeros(5), 1, [slice(2.625, 11.625)]).tolist()
[0.0, 0.0, 0.375, 1.0, 1.0]

kwimage.util_warp.subpixel_maximum(dst, src, index, interp_axes=None)

Take the max of the source values array into and the destination array at a particular subpixel index. Modifies the destination array.

Parameters:

• dst (ArrayLike) – destination array to index into

• src (ArrayLike) – source array that agrees with the index

• index (Tuple[slice]) – subpixel slice into dst that corresponds with src

• interp_axes (tuple) – specify which axes should be spatially interpolated

Example

>>> dst = np.array([0, 1.0, 1.0, 1.0, 0])
>>> src = np.array([2.0, 2.0])
>>> index = [slice(1.6, 3.6)]
>>> subpixel_maximum(dst, src, index)
>>> print(ub.urepr(dst, precision=2, with_dtype=0))
np.array([0. , 1. , 2. , 1.2, 0. ])

Example

>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> dst = torch.zeros((1, 3, 5, 5)) + .5
>>> src = torch.ones((1, 3, 3, 3))
>>> index = (slice(None), slice(None), slice(1.4, 4.4), slice(1.25, 4.25))
>>> subpixel_maximum(dst, src, index)
>>> print(ub.urepr(dst.numpy()[0, 0], precision=2, with_dtype=0))
np.array([[0.5 , 0.5 , 0.5 , 0.5 , 0.5 ],
          [0.5 , 0.5 , 0.6 , 0.6 , 0.5 ],
          [0.5 , 0.75, 1.  , 1.  , 0.5 ],
          [0.5 , 0.75, 1.  , 1.  , 0.5 ],
          [0.5 , 0.5 , 0.5 , 0.5 , 0.5 ]])

kwimage.util_warp.subpixel_minimum(dst, src, index, interp_axes=None)

Take the min of the source values array into and the destination array at a particular subpixel index. Modifies the destination array.

Parameters:

• dst (ArrayLike) – destination array to index into

• src (ArrayLike) – source array that agrees with the index

• index (Tuple[slice]) – subpixel slice into dst that corresponds with src

• interp_axes (tuple) – specify which axes should be spatially interpolated

Example

>>> dst = np.array([0, 1.0, 1.0, 1.0, 0])
>>> src = np.array([2.0, 2.0])
>>> index = [slice(1.6, 3.6)]
>>> subpixel_minimum(dst, src, index)
>>> print(ub.urepr(dst, precision=2, with_dtype=0))
np.array([0. , 0.8, 1. , 1. , 0. ])

Example

>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> dst = torch.zeros((1, 3, 5, 5)) + .5
>>> src = torch.ones((1, 3, 3, 3))
>>> index = (slice(None), slice(None), slice(1.4, 4.4), slice(1.25, 4.25))
>>> subpixel_minimum(dst, src, index)
>>> print(ub.urepr(dst.numpy()[0, 0], precision=2, with_dtype=0))
np.array([[0.5 , 0.5 , 0.5 , 0.5 , 0.5 ],
          [0.5 , 0.45, 0.5 , 0.5 , 0.15],
          [0.5 , 0.5 , 0.5 , 0.5 , 0.25],
          [0.5 , 0.5 , 0.5 , 0.5 , 0.25],
          [0.5 , 0.3 , 0.4 , 0.4 , 0.1 ]])

kwimage.util_warp.subpixel_slice(inputs, index)

Take a subpixel slice from a larger image. The returned output is left-aligned with the requested slice.

Parameters:

• inputs (ArrayLike) – data

• index (Tuple[slice]) – a slice to subpixel accuracy

Example

>>> # xdoctest: +REQUIRES(module:torch)
>>> import kwimage
>>> import torch
>>> # say we have a (576, 576) input space
>>> # and a (9, 9) output space downsampled by 64x
>>> ospc_feats = np.tile(np.arange(9 * 9).reshape(1, 9, 9), (1024, 1, 1))
>>> inputs = torch.from_numpy(ospc_feats)
>>> # We detected a box in the input space
>>> ispc_bbox = kwimage.Boxes([[64,  65, 100, 120]], 'ltrb')
>>> # Get coordinates in the output space
>>> ospc_bbox = ispc_bbox.scale(1 / 64)
>>> tl_x, tl_y, br_x, br_y = ospc_bbox.data[0]
>>> # Convert the box to a slice
>>> index = [slice(None), slice(tl_y, br_y), slice(tl_x, br_x)]
>>> # Note: I'm not 100% sure this work right with non-intergral slices
>>> outputs = kwimage.subpixel_slice(inputs, index)

Example

>>> inputs = np.arange(5 * 5 * 3).reshape(5, 5, 3)
>>> index = [slice(0, 3), slice(0, 3)]
>>> outputs = subpixel_slice(inputs, index)
>>> index = [slice(0.5, 3.5), slice(-0.5, 2.5)]
>>> outputs = subpixel_slice(inputs, index)

>>> inputs = np.arange(5 * 5).reshape(1, 5, 5).astype(float)
>>> index = [slice(None), slice(3, 6), slice(3, 6)]
>>> outputs = subpixel_slice(inputs, index)
>>> print(outputs)
[[[18. 19.  0.]
  [23. 24.  0.]
  [ 0.  0.  0.]]]
>>> index = [slice(None), slice(3.5, 6.5), slice(2.5, 5.5)]
>>> outputs = subpixel_slice(inputs, index)
>>> print(outputs)
[[[20.   21.   10.75]
  [11.25 11.75  6.  ]
  [ 0.    0.    0.  ]]]

kwimage.util_warp.subpixel_translate(inputs, shift, interp_axes=None, output_shape=None)

Translates an image by a subpixel shift value using bilinear interpolation

Parameters:

• inputs (ArrayLike) – data to translate

• shift (Sequence) – amount to translate each dimension specified by interp_axes. Note: if inputs contains more than one “image” then all “images” are translated by the same amount. This function contains no mechanism for translating each image differently. Note that by default this is a y,x shift for 2 dimensions.

• interp_axes (Sequence) – axes to perform interpolation on, if not specified the final n axes are interpolated, where n=len(shift)

• output_shape (tuple) – if specified the output is returned with this shape, otherwise

Note

This function powers most other functions in this file. Speedups here can go a long way.

Example

>>> inputs = np.arange(5) + 1
>>> print(inputs.tolist())
[1, 2, 3, 4, 5]
>>> outputs = subpixel_translate(inputs, 1.5)
>>> print(outputs.tolist())
[0.0, 0.5, 1.5, 2.5, 3.5]

Example

>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> inputs = torch.arange(9).view(1, 1, 3, 3).float()
>>> print(inputs.long())
tensor([[[[0, 1, 2],
          [3, 4, 5],
          [6, 7, 8]]]])
>>> outputs = subpixel_translate(inputs, (-.4, .5), output_shape=(1, 1, 2, 5))
>>> print(outputs)
tensor([[[[0.6000, 1.7000, 2.7000, 1.6000, 0.0000],
          [2.1000, 4.7000, 5.7000, 3.1000, 0.0000]]]])

kwimage.util_warp._padded_slice(data, in_slice, ndim=None, pad_slice=None, pad_mode='constant', **padkw)

Allows slices with out-of-bound coordinates. Any out of bounds coordinate will be sampled via padding.

Note

Negative slices have a different meaning here then they usually do. Normally, they indicate a wrap-around or a reversed stride, but here they index into out-of-bounds space (which depends on the pad mode). For example a slice of -2:1 literally samples two pixels to the left of the data and one pixel from the data, so you get two padded values and one data value.

Parameters:

• data (Sliceable[T]) – data to slice into. Any channels must be the last dimension.

• in_slice (Tuple[slice, …]) – slice for each dimensions

• ndim (int) – number of spatial dimensions

• pad_slice (List[int|Tuple]) – additional padding of the slice

Returns:

data_sliced: subregion of the input data (possibly with padding,

depending on if the original slice went out of bounds)

st_dimsa list indicating the low and high space-time coordinate

values of the returned data slice.

Return type:

Tuple[Sliceable, List]

Example

>>> data = np.arange(5)
>>> in_slice = [slice(-2, 7)]

>>> data_sliced, st_dims = _padded_slice(data, in_slice)
>>> print(ub.urepr(data_sliced, with_dtype=False))
>>> print(st_dims)
np.array([0, 0, 0, 1, 2, 3, 4, 0, 0])
[(-2, 7)]

>>> data_sliced, st_dims = _padded_slice(data, in_slice, pad_slice=(3, 3))
>>> print(ub.urepr(data_sliced, with_dtype=False))
>>> print(st_dims)
np.array([0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 0, 0, 0, 0, 0])
[(-5, 10)]

>>> data_sliced, st_dims = _padded_slice(data, slice(3, 4), pad_slice=[(1, 0)])
>>> print(ub.urepr(data_sliced, with_dtype=False))
>>> print(st_dims)
np.array([2, 3])
[(2, 4)]

kwimage.util_warp._ensure_arraylike(data, n=None)

kwimage.util_warp._rectify_slice(data_dims, low_dims, high_dims, pad_slice=None)

Given image dimensions, bounding box dimensions, and a padding get the corresponding slice from the image and any extra padding needed to achieve the requested window size.

Parameters:

• data_dims (tuple) – n-dimension data sizes (e.g. 2d height, width)

• low_dims (tuple) – bounding box low values (e.g. 2d ymin, xmin)

• high_dims (tuple) – bounding box high values (e.g. 2d ymax, xmax)

• pad_slice (List[int|Tuple]) – pad applied to (left and right) / (both) sides of each slice dim

Returns:

data_slice - low and high values of a fancy slice corresponding to

the image with shape data_dims. This slice may not correspond to the full window size if the requested bounding box goes out of bounds.

extra_padding - extra padding needed after slicing to achieve

the requested window size.

Return type:

Tuple

Example

>>> # Case where 2D-bbox is inside the data dims on left edge
>>> # Comprehensive 1D-cases are in the unit-test file
>>> data_dims  = [300, 300]
>>> low_dims   = [0, 0]
>>> high_dims  = [10, 10]
>>> pad_slice  = [(10, 10), (5, 5)]
>>> a, b = _rectify_slice(data_dims, low_dims, high_dims, pad_slice)
>>> print('data_slice = {!r}'.format(a))
>>> print('extra_padding = {!r}'.format(b))
data_slice = [(0, 20), (0, 15)]
extra_padding = [(10, 0), (5, 0)]

kwimage.util_warp._warp_tensor_cv2(inputs, mat, output_dims, mode='linear', ishomog=None)

implementation with cv2.warpAffine for speed / correctness comparison

On GPU: torch is faster in both modes On CPU: torch is faster for homog, but cv2 is faster for affine

Benchmark

>>> # xdoctest: +REQUIRES(module:torch)
>>> from kwimage.util.util_warp import *
>>> from kwimage.util.util_warp import _warp_tensor_cv2
>>> from kwimage.util.util_warp import warp_tensor
>>> import torch
>>> import numpy as np
>>> ti = ub.Timerit(10, bestof=3, verbose=2, unit='ms')
>>> mode = 'linear'
>>> rng = np.random.RandomState(0)
>>> inputs = torch.Tensor(rng.rand(16, 10, 32, 32)).to('cpu')
>>> mat = torch.FloatTensor([[2.5, 0, 10.5], [0, 3, 0], [0, 0, 1]])
>>> mat[2, 0] = .009
>>> mat[2, 2] = 2
>>> output_dims = (64, 64)
>>> results = ub.odict()
>>> # -------------
>>> for timer in ti.reset('warp_tensor(torch)'):
>>>     with timer:
>>>         outputs = warp_tensor(inputs, mat, output_dims, mode=mode)
>>>         torch.cuda.synchronize()
>>> results[ti.label] = outputs
>>> # -------------
>>> inputs = inputs.cpu().numpy()
>>> mat = mat.cpu().numpy()
>>> for timer in ti.reset('warp_tensor(cv2)'):
>>>     with timer:
>>>         outputs = _warp_tensor_cv2(inputs, mat, output_dims, mode=mode)
>>> results[ti.label] = outputs
>>> import itertools as it
>>> for k1, k2 in it.combinations(results, 2):
>>>     a = kwarray.ArrayAPI.numpy(results[k1])
>>>     b = kwarray.ArrayAPI.numpy(results[k2])
>>>     diff = np.abs(a - b)
>>>     diff_stats = kwarray.stats_dict(diff, n_extreme=1, extreme=1)
>>>     print('{} - {}: {}'.format(k1, k2, ub.urepr(diff_stats, nl=0, precision=4)))
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autompl()
>>> kwplot.imshow(results['warp_tensor(torch)'][0, 0], fnum=1, pnum=(1, 2, 1), title='torch')
>>> kwplot.imshow(results['warp_tensor(cv2)'][0, 0], fnum=1, pnum=(1, 2, 2), title='cv2')

kwimage.util_warp.warp_points(matrix, pts, homog_mode='divide')

Warp ND points / coordinates using a transformation matrix.

Homogoenous coordinates are added on the fly if needed. Works with both numpy and torch.

Parameters:

• matrix (ArrayLike) – [D1 x D2] transformation matrix. if using homogenous coordinates D2=D + 1, otherwise D2=D. if using homogenous coordinates and the matrix represents an Affine transformation, then either D1=D or D1=D2, i.e. the last row of zeros and a one is optional.

• pts (ArrayLike) – [N1 x … x D] points (usually x, y). If points are already in homogenous space, then the output will be returned in homogenous space. D is the dimensionality of the points. The leading axis may take any shape, but usually, shape will be [N x D] where N is the number of points.

• homog_mode (str) – what to do for homogenous coordinates. Can either divide, keep, or drop. Defaults do ‘divide’.

Retrns:

new_pts (ArrayLike): the points after being transformed by the matrix

Example

>>> from kwimage.util_warp import *  # NOQA
>>> # --- with numpy
>>> rng = np.random.RandomState(0)
>>> pts = rng.rand(10, 2)
>>> matrix = rng.rand(2, 2)
>>> warp_points(matrix, pts)
>>> # --- with torch
>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> pts = torch.Tensor(pts)
>>> matrix = torch.Tensor(matrix)
>>> warp_points(matrix, pts)

Example

>>> from kwimage.util_warp import *  # NOQA
>>> # --- with numpy
>>> pts = np.ones((10, 2))
>>> matrix = np.diag([2, 3, 1])
>>> ra = warp_points(matrix, pts)
>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> rb = warp_points(torch.Tensor(matrix), torch.Tensor(pts))
>>> assert np.allclose(ra, rb.numpy())

Example

>>> from kwimage.util_warp import *  # NOQA
>>> # test different cases
>>> rng = np.random.RandomState(0)
>>> # Test 3x3 style projective matrices
>>> pts = rng.rand(1000, 2)
>>> matrix = rng.rand(3, 3)
>>> ra33 = warp_points(matrix, pts)
>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> rb33 = warp_points(torch.Tensor(matrix), torch.Tensor(pts))
>>> assert np.allclose(ra33, rb33.numpy())
>>> # Test opencv style affine matrices
>>> pts = rng.rand(10, 2)
>>> matrix = rng.rand(2, 3)
>>> ra23 = warp_points(matrix, pts)
>>> rb23 = warp_points(torch.Tensor(matrix), torch.Tensor(pts))
>>> assert np.allclose(ra33, rb33.numpy())

kwimage.util_warp.remove_homog(pts, mode='divide')

Remove homogenous coordinate to a point array.

This is a convinience function, it is not particularly efficient.

SeeAlso:

cv2.convertPointsFromHomogeneous

Example

>>> homog_pts = np.random.rand(10, 3)
>>> remove_homog(homog_pts, 'divide')
>>> remove_homog(homog_pts, 'drop')

kwimage.util_warp.add_homog(pts)

Add a homogenous coordinate to a point array

This is a convinience function, it is not particularly efficient.

SeeAlso:

cv2.convertPointsToHomogeneous

Example

>>> pts = np.random.rand(10, 2)
>>> add_homog(pts)

Benchmark

>>> import timerit
>>> ti = timerit.Timerit(1000, bestof=10, verbose=2)
>>> pts = np.random.rand(1000, 2)
>>> for timer in ti.reset('kwimage'):
>>>     with timer:
>>>         kwimage.add_homog(pts)
>>> for timer in ti.reset('cv2'):
>>>     with timer:
>>>         cv2.convertPointsToHomogeneous(pts)
>>> # cv2 is 4x faster, but has more restrictive inputs

kwimage.util_warp.subpixel_getvalue(img, pts, coord_axes=None, interp='bilinear', bordermode='edge')

Get values at subpixel locations

Parameters:

• img (ArrayLike) – image to sample from

• pts (ArrayLike) – subpixel rc-coordinates to sample

• coord_axes (Sequence) – axes to perform interpolation on, if not specified the first d axes are interpolated, where d=pts.shape[-1]. IE: this indicates which axes each coordinate dimension corresponds to.

• interp (str) – interpolation mode

• bordermode (str) – how locations outside the image are handled

Example

>>> from kwimage.util_warp import *  # NOQA
>>> img = np.arange(3 * 3).reshape(3, 3)
>>> pts = np.array([[1, 1], [1.5, 1.5], [1.9, 1.1]])
>>> subpixel_getvalue(img, pts)
array([4. , 6. , 6.8])
>>> subpixel_getvalue(img, pts, coord_axes=(1, 0))
array([4. , 6. , 5.2])
>>> # xdoctest: +REQUIRES(module:torch)
>>> import torch
>>> img = torch.Tensor(img)
>>> pts = torch.Tensor(pts)
>>> subpixel_getvalue(img, pts)
tensor([4.0000, 6.0000, 6.8000])
>>> subpixel_getvalue(img.numpy(), pts.numpy(), interp='nearest')
array([4., 8., 7.], dtype=float32)
>>> subpixel_getvalue(img.numpy(), pts.numpy(), interp='nearest', coord_axes=[1, 0])
array([4., 8., 5.], dtype=float32)
>>> subpixel_getvalue(img, pts, interp='nearest')
tensor([4., 8., 7.])

References

[SO12729228]

stackoverflow.com/questions/12729228/simple-binlin-interp-images-numpy

SeeAlso:

cv2.getRectSubPix(image, patchSize, center[, patch[, patchType]])

kwimage.util_warp.subpixel_setvalue(img, pts, value, coord_axes=None, interp='bilinear', bordermode='edge')

Set values at subpixel locations

Parameters:

• img (ArrayLike) – image to set values in

• pts (ArrayLike) – subpixel rc-coordinates to set

• value (ArrayLike) – value to place in the image

• coord_axes (Sequence) – axes to perform interpolation on, if not specified the first d axes are interpolated, where d=pts.shape[-1]. IE: this indicates which axes each coordinate dimension corresponds to.

• interp (str) – interpolation mode

• bordermode (str) – how locations outside the image are handled

Example

>>> from kwimage.util_warp import *  # NOQA
>>> img = np.arange(3 * 3).reshape(3, 3).astype(float)
>>> pts = np.array([[1, 1], [1.5, 1.5], [1.9, 1.1]])
>>> interp = 'bilinear'
>>> value = 0
>>> print('img = {!r}'.format(img))
>>> pts = np.array([[1.5, 1.5]])
>>> img2 = subpixel_setvalue(img.copy(), pts, value)
>>> print('img2 = {!r}'.format(img2))
>>> pts = np.array([[1.0, 1.0]])
>>> img2 = subpixel_setvalue(img.copy(), pts, value)
>>> print('img2 = {!r}'.format(img2))
>>> pts = np.array([[1.1, 1.9]])
>>> img2 = subpixel_setvalue(img.copy(), pts, value)
>>> print('img2 = {!r}'.format(img2))
>>> img2 = subpixel_setvalue(img.copy(), pts, value, coord_axes=[1, 0])
>>> print('img2 = {!r}'.format(img2))

kwimage.util_warp._bilinear_coords(ptsT, impl, img, coord_axes)