Features

Classes for encapsulatng and modifying feature vectors.

FeatureExtractor

class perception.FeatureExtractor

Bases: object

extract(image)

Returns a set of extracted features for an image :param image: object to extract for :type image: Image

CNNBatchFeatureExtractor

class perception.CNNBatchFeatureExtractor(config)

Bases: perception.feature_extractors.FeatureExtractor

Extract feature descriptors for images in a giant batch using Convolutional Neural Networks.

cnn

AlexNet – the convolutional neural network to use

open()

Opens the tensorflow session. For memory management.

close()

Closes the tensorflow session. For memory management.

extract(images)

Form feature descriptors for a set of images.

Parameters:images (list of :obj`Image` or :obj`numpy.ndarray`) – images to extract features for

CNNReusableBatchFeatureExtractor

class perception.CNNReusableBatchFeatureExtractor(cnn)

Bases: perception.feature_extractors.CNNBatchFeatureExtractor

Extract feature descriptors for images in a giant batch. Allows you to initialize the extractor with a pre-existing CNN, for memory management reasons.

Feature

class perception.Feature

Bases: object

Abstract class for features

BagOfFeatures

class perception.BagOfFeatures(features=None)

Wrapper for a list of features, created for the sake of future bag-of-words reps.

features

list of Feature – list of feature objects

add(feature)

Add a new feature to the bag.

Parameters:feature (Feature) – feature to add
extend(features)

Add a list of features to the bag.

Parameters:feature (list of Feature) – features to add
feature(index)

Returns a feature.

Parameters:index (int) – index of feature in list
Returns:
Return type:Feature
feature_subset(indices)

Returns some subset of the features.

Parameters:indices (list of int) – indices of the features in the list
Returns:
Return type:list of Feature
descriptors

Make a nice array of the descriptors

reference_frames

Make a nice array of the reference frames

keypoints

Make a nice array of the keypoints

normals

Make a nice array of the normals

LocalFeature

class perception.LocalFeature(descriptor, rf, point, normal)

Bases: perception.features.Feature

Local (e.g. pointwise) features on shape surfaces.

descriptor

numpy.ndarray – vector to describe the point

reference_frame

numpy.ndarray – reference frame of the descriptor, as an array

point

numpy.ndarray – 3D point on shape surface that descriptor corresponds to

normal

numpy.ndarray – 3D surface normal on shape surface at corresponding point

GlobalFeature

class perception.GlobalFeature(key, descriptor, pose=None)

Bases: perception.features.Feature

Global features of a full shape surface.

key

str – object key in database that descriptor corresponds to

descriptor

numpy.ndarray – vector to describe the object

pose

autolab_core.RigidTransform – pose of object for the descriptor, if relevant

SHOTFeature

class perception.SHOTFeature(descriptor, rf, point, normal)

Bases: perception.features.LocalFeature

Signature of Oriented Histogram (SHOT) features

MVCNNFeature

class perception.MVCNNFeature(key, descriptor, pose=None)

Bases: perception.features.GlobalFeature

Multi-View Convolutional Neural Network (MV-CNN) descriptor

Correspondences

class perception.Correspondences(index_map, source_points, target_points)

Wrapper for point-set correspondences.

index_map

list of int – maps list indices (source points) to target point indices

source_points

Nx3 numpy.ndarray – set of source points for registration

target_points

Nx3 numpy.ndarray – set of target points for registration

num_matches

int – the total number of matches

NormalCorrespondences

class perception.NormalCorrespondences(index_map, source_points, target_points, source_normals, target_normals)

Bases: perception.feature_matcher.Correspondences

Wrapper for point-set correspondences with surface normals.

index_map

list of int – maps list indices (source points) to target point indices

source_points

Nx3 numpy.ndarray – set of source points for registration

target_points

Nx3 numpy.ndarray – set of target points for registration

source_normals

normalized Nx3 numpy.ndarray – set of source normals for registration

target_normals

normalized Nx3 numpy.ndarray – set of target points for registration

num_matches

int – the total number of matches

FeatureMatcher

class perception.FeatureMatcher

Bases: object

Generic feature matching between local features on a source and target object using nearest neighbors.

static get_point_index(point, all_points, eps=0.0001)

Get the index of a point in an array

match(source_obj, target_obj)

Matches features between a source and target object. Source and target object types depend on subclass implementation.

RawDistanceFeatureMatcher

class perception.RawDistanceFeatureMatcher

Bases: perception.feature_matcher.FeatureMatcher

match(source_obj_features, target_obj_features)

Matches features between two graspable objects based on a full distance matrix.

Parameters:
  • source_obj_features (BagOfFeatures) – bag of the source objects features
  • target_obj_features (BagOfFeatures) – bag of the target objects features
Returns:

corrs – the correspondences between source and target

Return type:

Correspondences

PointToPlaneFeatureMatcher

class perception.PointToPlaneFeatureMatcher(dist_thresh=0.05, norm_thresh=0.75)

Bases: perception.feature_matcher.FeatureMatcher

Match points using a point to plane criterion with thresholding.

dist_thresh

float – threshold distance to consider a match valid

norm_thresh

float – threshold cosine distance alignment betwen normals to consider a match valid

match(source_points, target_points, source_normals, target_normals)

Matches points between two point-normal sets. Uses the closest ip to choose matches, with distance for thresholding only.

Parameters:
  • source_point_cloud (Nx3 numpy.ndarray) – source object points
  • target_point_cloud (Nx3 numpy.ndarray) – target object points
  • source_normal_cloud (Nx3 numpy.ndarray) – source object outward-pointing normals
  • target_normal_cloud (Nx3 :obj`numpy.ndarray`) – target object outward-pointing normals
Returns:

the correspondences between source and target

Return type:

obj`Correspondences`