Features¶

Classes for encapsulatng and modifying feature vectors.

FeatureExtractor¶

class perception.FeatureExtractor¶

Bases: object

extract(image)¶: Returns a set of extracted features for an image :param image: object to extract for :type image: Image

CNNBatchFeatureExtractor¶

class perception.CNNBatchFeatureExtractor(config)¶

Bases: perception.feature_extractors.FeatureExtractor

Extract feature descriptors for images in a giant batch using Convolutional Neural Networks.

cnn¶: AlexNet – the convolutional neural network to use

open()¶: Opens the tensorflow session. For memory management.

close()¶: Closes the tensorflow session. For memory management.

extract(images)¶

Form feature descriptors for a set of images.

Parameters:	images (`list` of :obj`Image` or :obj`numpy.ndarray`) – images to extract features for

CNNReusableBatchFeatureExtractor¶

class perception.CNNReusableBatchFeatureExtractor(cnn)¶

Bases: perception.feature_extractors.CNNBatchFeatureExtractor

Extract feature descriptors for images in a giant batch. Allows you to initialize the extractor with a pre-existing CNN, for memory management reasons.

Feature¶

class perception.Feature¶

Bases: object

Abstract class for features

BagOfFeatures¶

class perception.BagOfFeatures(features=None)¶

Wrapper for a list of features, created for the sake of future bag-of-words reps.

features¶: list of Feature – list of feature objects

add(feature)¶

Add a new feature to the bag.

Parameters:	feature (`Feature`) – feature to add

extend(features)¶

Add a list of features to the bag.

Parameters:	feature (`list` of `Feature`) – features to add

feature(index)¶

Returns a feature.

Parameters:	index (int) – index of feature in list
Returns:
Return type:	`Feature`

feature_subset(indices)¶

Returns some subset of the features.

Parameters:	indices (`list` of `int`) – indices of the features in the list
Returns:
Return type:	`list` of `Feature`

descriptors¶: Make a nice array of the descriptors

reference_frames¶: Make a nice array of the reference frames

keypoints¶: Make a nice array of the keypoints

normals¶: Make a nice array of the normals

LocalFeature¶

class perception.LocalFeature(descriptor, rf, point, normal)¶

Bases: perception.features.Feature

Local (e.g. pointwise) features on shape surfaces.

descriptor¶: numpy.ndarray – vector to describe the point

reference_frame¶: numpy.ndarray – reference frame of the descriptor, as an array

point¶: numpy.ndarray – 3D point on shape surface that descriptor corresponds to

normal¶: numpy.ndarray – 3D surface normal on shape surface at corresponding point

GlobalFeature¶

class perception.GlobalFeature(key, descriptor, pose=None)¶

Bases: perception.features.Feature

Global features of a full shape surface.

key¶: str – object key in database that descriptor corresponds to

descriptor¶: numpy.ndarray – vector to describe the object

pose¶: autolab_core.RigidTransform – pose of object for the descriptor, if relevant

SHOTFeature¶

class perception.SHOTFeature(descriptor, rf, point, normal)¶

Bases: perception.features.LocalFeature

Signature of Oriented Histogram (SHOT) features

MVCNNFeature¶

class perception.MVCNNFeature(key, descriptor, pose=None)¶

Bases: perception.features.GlobalFeature

Multi-View Convolutional Neural Network (MV-CNN) descriptor

Correspondences¶

class perception.Correspondences(index_map, source_points, target_points)¶

Wrapper for point-set correspondences.

index_map¶: list of int – maps list indices (source points) to target point indices

source_points¶: Nx3 numpy.ndarray – set of source points for registration

target_points¶: Nx3 numpy.ndarray – set of target points for registration

num_matches¶: int – the total number of matches

NormalCorrespondences¶

class perception.NormalCorrespondences(index_map, source_points, target_points, source_normals, target_normals)¶

Bases: perception.feature_matcher.Correspondences

Wrapper for point-set correspondences with surface normals.

index_map¶: list of int – maps list indices (source points) to target point indices

source_points¶: Nx3 numpy.ndarray – set of source points for registration

target_points¶: Nx3 numpy.ndarray – set of target points for registration

source_normals¶: normalized Nx3 numpy.ndarray – set of source normals for registration

target_normals¶: normalized Nx3 numpy.ndarray – set of target points for registration

num_matches¶: int – the total number of matches

FeatureMatcher¶

class perception.FeatureMatcher¶

Bases: object

Generic feature matching between local features on a source and target object using nearest neighbors.

static get_point_index(point, all_points, eps=0.0001)¶: Get the index of a point in an array

match(source_obj, target_obj)¶: Matches features between a source and target object. Source and target object types depend on subclass implementation.

RawDistanceFeatureMatcher¶

class perception.RawDistanceFeatureMatcher¶

Bases: perception.feature_matcher.FeatureMatcher

match(source_obj_features, target_obj_features)¶

Matches features between two graspable objects based on a full distance matrix.

Parameters:	source_obj_features (`BagOfFeatures`) – bag of the source objects features target_obj_features (`BagOfFeatures`) – bag of the target objects features
Returns:	corrs – the correspondences between source and target
Return type:	`Correspondences`

PointToPlaneFeatureMatcher¶

class perception.PointToPlaneFeatureMatcher(dist_thresh=0.05, norm_thresh=0.75)¶

Bases: perception.feature_matcher.FeatureMatcher

Match points using a point to plane criterion with thresholding.

dist_thresh¶: float – threshold distance to consider a match valid

norm_thresh¶: float – threshold cosine distance alignment betwen normals to consider a match valid

match(source_points, target_points, source_normals, target_normals)¶

Matches points between two point-normal sets. Uses the closest ip to choose matches, with distance for thresholding only.

Parameters:	source_point_cloud (Nx3 `numpy.ndarray`) – source object points target_point_cloud (Nx3 `numpy.ndarray`) – target object points source_normal_cloud (Nx3 `numpy.ndarray`) – source object outward-pointing normals target_normal_cloud (Nx3 :obj`numpy.ndarray`) – target object outward-pointing normals
Returns:	the correspondences between source and target
Return type:	obj`Correspondences`