A method is presented for calibrating the CyberGlove using a vision based system. Coloured dots are manually placed on the joints and the joint angles are then calculated by analysing the video. The gain and offset are then calculated using linear regression when there is a one to one relationship between the actual joint angle a and the CyberGlove reading, or a least squares method when there is a one to two relationship (that is, two CyberGlove readings can affect a single actual joint angle). The system provides good results when visually inspected.

An algorithm is presented for reconstructing the position of missing markers (for example when using the OptoTrak) based on knowledge of the distances between the markers.

A method is presented for calibrating the Cyberglove without requiring additional hardware. The subject holds the thumb and the index (or other) finger together, while moving the other joints. The hand is then approximated as a close kinematic chain, and the ratios of the bone lengths are assumed, and hence the only unknown joint angle is that between the thumb and finger. A least squares fit is used to minimize the calculated distance between the two fingers. The parameters in the optimization are the sensor gains, fixed offsets, the bone lengths and cross coupling terms (which allow for a joint angles to be dependent on the value of more than one sensor). Limits on parameter deviation are necessary to prevent trivial solutions.

A method is presented for finding the centre of rotation or axis of rotation for a joint from the positions of 3D markers (eg Optotrak). Unlike other methods, the relationship between the markers is not assumed - each marker only need maintain a constant distance from the centre/axis of rotation. The solution is a least squares solution and does not require setting any parameters.

A series of experiments were performed to test the accuracy of the CyberGlove. They found that noise had an insignificant effect on the results. They showed that using the standard calibration (and not calibrating for each user), reasonable differentiation could be made between a small number of divisions. For better differentiating capabilities, it is necessary to calibrate for an individual user - this can be achieved by measuring the data from the glove at known joint angles.

A method for calibrating an instrumented glove (the UTAH Dextrous Hand Master) is presented. The finger is modeled by an open-link kinematic chain. The end point of this kinematic chain is externally measured (using an Optotrak marker). Singular Value Decomposition is used to find the parameters. The poses used are carefully selected to use most of the joint ranges, and parameter scaling is used in the optimization procedure.

A model of the location of the major flesh points in the mouth is presented for speech production. The effect of coarticulation (where the position of a point during speech is affected by the previous or following phoneme) is estimated by measuring the amount of effort, defined to be the local curvature of the trajectory. The inclusion of coarticulation in their model produced better predictions.

The differences in playing on the piano two sequences that begin the same but have significant differences at some point were compared. It was found that in some cases there is anticipatory changes in the kinematics about one note before the divergence. As this was only sometimes the case, they concluded that such movements are executed in a strictly serial ordering as long as this is compatible with the task.

Coarticulation in ASL fingerspelling was studied. The joint angles of the hand were measured and discriminant analysis was used to classify postures and define a coarticulation measure. Substantial evidence was found of both assimilation (reducing distances between two shapes) and dissimilation (emphasizing the difference between shapes) - these effects were even found to apply concurrently in different joint angles. Assimilation was primarily found in the thumb and wrist joints, while dissimilation was found mainly in the PIP of the index and middle fingers. This discrimination may aid in recognition, as these joints have been found to be sufficient for 88% correct classification of letters. They suggest that the concurrent instances of assimilation and dissimilation argue against synergistic control, however, they reconcile this with the evidence for synergies by hypothesizing that there is a combination of a general tendency for coordination of all fingers with an ability for individuated control.

For recognizing an alphabet for fingerspelling, PCA is compared to using a subset of the joint angles. It was found that using a subset of joint angles was superior to using a similar sized PCA weighting vector in transmitting information about the pose. They suggest that hence synergies are not used as a primary control strategy for this task, and that recognition can be performed more easily and with less measured angles using this technique rather than PCA.

Anticipation in handwriting was studied by presenting to subjects the middle letter in a trigram and asking them to predict what the third letter will be (from two choices). Subjects could predict this with a reasonable accuracy (around 60%), however when the ratio between the radius of curvature and tangential velocity (ie the two thirds power law) is changed, the rate of accuracy drops significantly. They suggest that the ability to predict the next letter is based on an internal model of the movement. They also note that the prediction is based on the kinematic properties rather than the shape.

Neural networks are constructed for learning the joint angles of the index finger, based on the normalized number of EMG zero crossings, and a torque estimate based on the EMG (which is a combination of a number of EMG signals). Both methods modeled well the joint angles of the finger. Although considering the torque estimates rather than EMG was supposed to allow for different muscle cocontraction strategies, both methods performed statistically the same. It is hypothesized that the neural network based on EMG learned the cocontraction of parts of the movement.

This article presents an analysis of grasping by a multi-fingered hand and considers what ``intelligence'' is necessary for successful grasping. The intelligence is based on what prior knowledge is necessary of the object (e.g. only kinematic information, information about the mass, etc). It is suggested that successful grasping can be achieved with little intelligence by using a combination of learned signals combined with sensed physical values. For secure grasping, force feedback is not necessary, however for artificial grasping such feedback can be used instead of the learning process that humans may perform.

This study compared the effects of different tasks performed on an object with the performance of the reach and grasp movement before performing the task. Based on measuring movement parameters such as movement time, velocity and acceleration, they observed that the task to be performed does affect the prehension movement leading up to it.

A survey is made of the requirements of a machine hand for grasping, looking at human operability, manipulator dexterity and grasping robustness.

The influence of obstacles of grasping movement time (thumb / index finger) was considered for different models. It was concluded that their influence is best described by a model based on control of the thumb/index fingers, rather than a limitation of grip aperture.

An fMRI study was performed where subjects observed pictures of objects and had to decide in a forced choice task whether to use the object or grasp it. Use the object was divided into prehensile use (pinch or clench) or non-prehensile use (palm or poke), while the grasp condition was either pinch or clench. The left inferior frontal gyrus, posterior superior temporal gyrus and inferior parietal lobule (IPL) showed significantly greater activation in non-prehensile use compared to grasp. No areas were observed that showed greater activation for grasp. They suggest that this might be because computations for object grasping are a subset of the computation for using. A difference was only seen in the left IPL when comparing non-prehensile use and prehensile use. They conclude that the left IPL is important for storing knowledge of hand postures for functional object use.

In this work, a cybernetic hand, called the ``cyberglove'' is presented. The cyberglove has 6 actuators (motors), controlling the four fingers independently and the thumb. Each of the four fingers has three joints which are controlled by one ``tendon''. The thumb is controlled by two motors. The hand is able to perform opposition with the thumb, and can perform lateral pinch, cylindrical, spherical and tripod grasps. The high level control (i.e., selection of which grasp and amount of force) will eventually be based on EEG / EMG signals. The low level control is responsible for actuating the desired force. Some sensory feedback is also collected.

A set of visually computable grasp features was presented such as contact point arrangement and force equilibrium. They were used to build a neural network to predict whether a grasp will be stable. The training was performed by a robot which shook the objects to test their stability. This method does not require a model of the object to be grasped.

A measure is presented for task compatibility of a manipulator for certain task requirements (in terms of effecting or controlling velocity and force). The measure is based on the velocity and force ellipsoids. The transmission ratio of applied force or velocity in terms of joint coordinates to the same quantity in task coordinates in computed. The transmission ratios represent the amplification in force and velocity, while accuracy is represented by the reciprocals of these ratios. The compatibility index is based on summing the ratios or their reciprocal (depending on the task) in the appropriate direction.

A set of experiments were performed to test whether grasps are planned by generation or by recall. The experiment involved grasping and moving a cylinder to different heights. Initially, the postures selected were assumed to be such that at the end of the movement, the joints will be in mid-range (the end-state comfort effect). This means that the higher the position to which it will be moved, the lower it will be grasped. In movements that began where the previous movement ended to where the previous movement started, the initial posture was close to where they had grasped it previously at the end of the movement. From this finding, they suggest that movement plans are recalled as well as being generated.

This paper examined the relationship between the orientation and shape (different aspect ratios) of a cylinder, and the orientation and aperture of the grasping hand (consisting of precision grasps on the index finger and thumb). They found that the orientation of grasping was such that the cylinder was grasped (close to) along its principle axes, with more (68%) along the minor axis. It should be noted that grasping along one of the principal axes is the only stable grasp of a cylinder. They also found that the final hand orientation could be inferred after only 30% of the movement distance, whereas for the aperture this was only possible after 80% of the movement distance. They conclude that the findings confer with the theory that only the appropriate visual quantities are used in planning a movement, and that the errors observed are consistent with those due to the grasp being planned incorrectly due to a distorted perception of the cylinder's shape.

A review is presented of studies in human grasp choice, and analytical methods used for robotic grasping. Various categorizations and taxonomies used for human grasping are described, as well as expert systems. Algorithms for robotic grasp planning, including limitations of such models due to the assumptions made are presented. Different features that are optimized or used as constraints are described, and finally comments are made on the connections between the two.

For force production task, they define a mode, which is the forces produced by all the fingers as a result of voluntary force production in one finger. Multiple finger force production can be modeled by the superposition of modes but with a weight dependent on the the number of fingers used (to take into account force deficit). This model captures the behaviour of the enslaving effect for multiple fingers.

The ability to combine internal models for grasping was examined by measuing the peak grip force rate for lifting objects of the same appearance but different weight and their combination. In contrast to other studies, it was found that subjects could learn grip force scaling for two seperate weights simultaneously even when alternating between them. They suggest that this difference was because the objects were clearly distinct in the environment. In addition, they suggest the CNS may be able to additively combine two dynamic internal models to determine the necessary grip force for lifting the two objects together. It appeared that the subjects acted in a Bayesian way to deal with the uncertainty of the weights when they were combined.

A formalism is described for compliant motion, as an extension of Mason's hybrid control. It consists of selection of the task frame relative to the end effector, constraints on the force, velocity or tracking (detection of errors based on forces or velocities) in 6 dimensions in the task frame, additional task frame or end effector motion constraints, feedforward velocity constraints and task termination conditions.

A framework for implementing compliant robot motion is presented. The system receives as input the constraints as described in a previous work. It is based on a multidimensional position control loop embedded in a multidimensional force control loop.

A method of learning human grasps for telerobotics is presented. Four human grasps are recognized using magnetic trackers placed on four fingers. A Hidden Markov Model (HMM) is used for grasp recognition. The human posture is mapped to a (simulated) robotic posture using a trained artificial neural network.

The difference in grasping an object with different visual feedback was studied. The subjects were shown, before the movement, either the real object, a virtual computer rendered object or a symbolic presentation (using a mirror setup). The visual information was removed at the initiation of the movement. Haptic feedback was provided (using a real object). Different kinematic properties were compared (e.g. preshape aperture, grasp onset latency, movement velocity), and no significant difference was seen between grasping real and virtual objects (as opposed to pantomiming behaviour found in other studies).

A grasping experiment under the Ebbinghaus illusion showed that contrary to previous studies, the illusion affects grasping to the same extent as perception. It was shown that an alternate hypothesis of object avoidance cannot explain the results. They suggest that the same source is responsible for the illusion in both perception and in grasping. This reduces the evidence for a perception vs action hypothesis of brain organization.

When passively observed natural reaching and grasping movements, profiles of cortical excitability were in concordance with the kinematic profiles of the movements, and evoked greater corticospinal facilitation than the observation of unnatural movements. Depending on the type of perturbations, either no modulation was observed, or one similar to the natural movement. It is thus suggested that the resonant motor plan is loaded at the beginning and tends to complete itself regardless of changes in visual cues.

Three types of neural networks were compared for predicting the finger forces required in a torque stabilization experiment. The most effective one was a hierarchical two layer network, where first the virtual finger force was calculated, then in the second layer the finger forces were calculated. Input to the first layer was also available to the second layer. The performance was better than for a classical 3-layer network. They suggest that this supports the notion of hierarchical control of prehension.

The control of the fingers in grasping a sphere was studied under different conditions (varying the distance to the object and its size). Most of the time a tripod grasp was selected. This grasp consists of apeture components - the opening of the thumb and index/middle fingers which was coordinated, and a seperation component (between the thumb and index fingers) which was weakly coupled with the aperture component. They relate these findings to the use of the virtual fingers to form the grasp - one for the thumb and the other for the other finger(s).

The differences in grasping an object with the hand and with a tool (two mechanized fingers) were studied. Some kinematic features were preserved, while others were different. In particular, the same finger pre-shape was used for the grasp in both cases, but the temporal pattern of the movement was different (a pronounced velocity plateau, shorter opening phase and longer closure phase). Based on these results, they suggest that some grasp features are encoded independently of the effector used.

The increase in grip force observed in older adults may be due to the lower coefficient of friction of their skin rather than to compensate for greater instability.

The effect of displaying the word LARGE or SMALL on a block being grasped in a reach-to-grasp movement was studied. It was found that an effect was seen in the early stages of the movement, but the effect was seen less towards the conclusion of the movement. An explanation for this behaviour was that the meaning of the word affected the early planning stages of the movement, but do not affect the on-line control which uses different information.

Words representing large objects (such as apple) and small objects (such as grape) were presented to subjects before a grasping movement. Words representing large objects led to a larger grip aperture. The interference was apparent early in the movement and its effect diminished as the hand approached the target, which they explain as the result of on-line correction of the semantic effect. They consider this behaviour in terms of the distinction between motor planning and on-line control.

Precision grasp-like movements with the thumb and index finger were performed, and the joint angles, velocities and acceleration were measured with the CyberGlove. Inverse dynamics were then performed to estimate the joint torques, on which they performed PCA to joint torque synergies. Although the Principal Components for torque described less variance that those for joint angles, under different conditions (faster or slower velocity), the joint torques were observed to scale linearly with the velocity.

Muscle synergies during precision grip in humans was studied by looking at the EMG. It was found that rather than using a unique muscle synergestic muscle activation pattern for a particular task, the CNS appears to use flexible short-term muscle synergies. This variation does not explain the consistent and accurate behaviour observed in such grips.

A model for robot grasping is presented. Three different optimization criteria are suggested for producing high-quality grasps - minimizing muscle effort, minimizing the maximum applied finger forces (to prevent object damage), and maximizing the degree of uniformity between the fingers. By solving these optimization problems, suggested grips can be produced.

Three grasping quality measures are suggested - minimal muscle effort, minimum of the maximum applied finger forces, and minimizing an entropy-like function (which causes a uniform level of the contact forces). Subjects were asked to grip various objects, and the numerical values for these quality measures were calculated. These were compared with the subjects' perceived quality of the grip using a psychophysical magnitude estimation method. It was found that the measure of the uniform level of contact forces is dominant in the human quality sense.

This paper explores the paradox of separate channels for reaching and grasp formation and a holistic programming of such movements. To combine the two notions, it is suggested that the movements are organized on several levels. The individual channels are embedded into an internal model of the entire movement which exerts top-down control.

Grasp stability (of a multi-fingered hand) is examined by expressing it in terms of differential equations. The stability of the grasps is then determined by considering the Liapunov stability of the system of differential equations. Methods are then given for synthesizing stable grasps based on these concepts.

The coordination of hand movements and gaze was studied. Subjects fixated on on landmarks critical for control of the task, such as points where contact was made with the object. They did not fixate on the arm or the bar being grasped. They concluded that gaze supports the planning of the task by fixating on key points.

The control of grip stability was studied during pronation and supination movements of an object which has destabilizing torque dependent on the angle of rotation. It was found that the grip force for stabilizing the object increased directly with the destabilizing torque. As blocking sensory information from the fingertips did not significantly change the coordination, they concluded that feed-forward rather than feedback mechanisms are responsible for grip force control.

An algorithm is presented for learning to grasp using visual information based on a heuristic. Learning is used to improve the estimation of where to grasp and well as the measures of grasp quality.

The trajectories of the fingertips during grasping of 5 objects was studied. A good fit of the fingertip positions was found to a logarithmic spiral in the theta-r plane (and better than a polynomial in the x-y plane). The spiral was a good fit regardless of starting posture. More variance was seen for the thumb than the other fingers. Sometimes highly linear relationships were found between joint angles although not consistently. The lack of correlation found may be because the correlation is piece-wise rather than consistent over the movement.

The UCM approach is applied to a difficult multi-finger ramp force production task. The contributions of forces that contribute to the task force, and of moments in the frontal plane were considered as the hypotheses. The variance was partitioned into the component which does not affect the hypotheses (UCM) and the component that does. No difference was seen in the variance of the forces before learning, but a significant difference was seen in the variance after learning (i.e. less variance in the task component). The variance in the moment stabilization became worse after learning (this is an unavoidable consequence of better force stabilization).

The properties of the grasp stiffness matrix are examined. It is shown that a stiffness matrix is conservative if the matrix is symmetric and satisfies a certain differential condition. In general a conservative stiffness matrix is Cartesian space will be nonconservative when transformed into joint space using a configuration dependent Jacobian (and vice versa).

Three issues involving multifingered hands were examined. A method is presented for selecting internal grasp forces to produce a stable grasp. It is based on specifying suitable constraints (e.g. friction, joint torque limits) and finding the configuration that is furthest from violating any of these constraints. Also presented is a method for finding motion of the fingertips (e.g. rolling) due to movement of the object. Finally, a method is presented for finding the workspace of a hand/object pair, that is, the range of manipulators for a particular configuration of contact points on the object and locations of the contact points on the fingertips.

A non-dimensionalized composite grasp index was constructed, based on a stability index, a grasp uncertainty index, a maximum force transmission ratio index, a task isotropy index, and a stiffness mapping-based isotropy index. Each index was appropriately normalized, and has a weighting factor based on the relative importance given to this component. By altering the weighting factors for each index in a simulation, different optimal grasps were produced.

In order to determine the optimal grasp, a series of performance indices were defined. These indices are a stability grasp index (how close the grasp points are to a regular polygon), an uncertainty grasp index (how far away the grasp points are from edges), a maximum force transmission ratio index (based on the force ellipsoid and the desired force direction), a task isotropy index (distance from singularities) and a stiffness mapping-based grasp isotropy index (based on the grasp stiffness). These measures are normalized (by dividing them by the difference between the maximum and minimum possible values) and thus also non-dimensional. Different weights can be given to the different indices depending on the task.

A method is described for planning the necessary stiffness for various grasping and manipulation tasks. The stiffness of the grasped object is related to the stiffness of the joints through the grasp matrix. The desired stiffness geometry for the task in object coordinates can then be transformed to determine the necessary joint stiffness and/or geometry of the hand. Various examples are given.

A pneumatically controlled glove is described that can be used for rehabilitation. The five fingers of the glove, which is worn by the subject, can be extended together by the bladder which is sewn onto the palm side of the glove. The pressure of the bladder is controlled by a servo valve connected to a computer. Its use was demonstrated on a stroke survivor in a virtual reality simulation where the patient has to grasp objects, assisted by the glove.

The effect of semantic category (living vs non-living objects) and size on a reach-to-grasp task was examined. Inconsistent results were found regarding the difference in speed between living and non-living objects, but the size was found to have a significant effect on the kinematic parameters. The effects of distractors was also noted.

The adaption to a novel, velocity dependent force perturbation was found to be different depending on the specified goal. When subjects were asked to perform a spatial goal (continue to the target), their movements became curved but returned to reach the final point. In constrast, when subjects were asked to maintain the same effort, the deviation increased throughout the movement, resulting in large endpoint deviations. A significant after effect was only seen with the spatial goal.

Synergies have been observed for finger force production, that is, that other fingers will compensate for an error or variation in the force produced by one finger. This studied asked the question of whether the other fingers can correct for timing errors, i.e. if there are timing synergies. Evidence was not found for such synergies, rather, if one finger sped up, the others were also likely to speed up.

An algorithm is presented for learning hand gestures using a Hidden Markov Models (HMMs). Twenty joint angles from the hand are used as input. They are first preprocessed by dividing them into gestures, resampling, applying a FFT and creating a single vector from the data. This is used as the input to the HMMs - there is one for each of an alphabet of gestures, and the one with the highest probability is selected if the classification is strong enough).

PCA is used to find a lower dimensional representation of static grasp postures using 18 joint angles of the fingers. They also consider a non-linear PCA, which allows non-linear mappings between the principal components and the desired posture. This produced slightly better results than the regular PCA, but is a more complex procedure.

A neural network was constructed that could predict the effects of force production in multi-fingered force production tasks. Unlike optimization techniques, this model accounts for force deficit and enslaving observed experimentally.

Several quality measures are defined for multi-fingered grasps. They present quality measures based on the grasp matrix, G. They introduce general quality measures, based on the smallest singular value of G, and the volume in wrench space. They also define a task-oriented quality measure, based on the task ellipsoid (force ellipsoid). The specification of the task ellipsoid for a task is based on experience with the task and similar tasks.

A frame invariant measure is defined for compliance grasps, and an interpretation of the stiffness matrix is given.

A method is presenting for calculating the stiffness matrix using the Hertz model. They contrast this to the linear spring compliance model that is commonly used but is not supported by experiments, and the coefficients must be determined experimentally.

To examine what is learnt during manipulation of unstable objects, an experiment was performed where the subjects had to balance a simulated inverted pendulum. When the arm posture was changed, the results were better when the effects of arm torque were matched to the first condition. From this result, they suggest that the subjects learnt the necessary joint torques rather than a general model of forces. A further experiment found that the advantage of training was object specific, based on comparing two different tasks with similar forces but different visual cues and requirements.

The relationship between the joint angles in the arm were studied during a reaching and grasping task of an object at different orientations. A linear relationship was observed between upper arm torsion and the torsion of the forearm relative to the upper arm. They conclude that a combination of upper arm, forearm and fingers are used to specify the orientation rather than by using separate transport and hand orientation components.

The difference in some kinematic parameters of the hand during different tasks was examined. Significant differences were seen, mainly in the relative time of the peak velocity of the wrist. The tasks that required greater precision has a longer deceleration phase. Based on these findings, they suggest that movement planning be relatively specific to the task.

A technique is described for modeling the PIJ joint angle and the forces applied at the finger tip (normal and shear forces but not moments) based on the patterns of blood volume beneath the fingernail. The blood volume is measured using LEDs and photo detectors. Shear forces are measured to an accuracy of 0.5N root mean square (rms) error, normal forces with 1N rms error and PIJ angles with 10 degrees rms error.

In the first part of this book, Mason analyses different types of contacts, using the notation of screws, twists and wrenches. He uses this to define which hand grips are stable. The grip transform is introduced as a way of transforming forces applied by the fingers to the force applied to the object. Stiffness control as a way of controlling the hand is also presented, as well as the design of a robotic hand (the Stanford/JPL hand). The second part of the book by Salisbury looks at the mechanics of grasping and pushing.

A system for combining several modes of communication for instructing a robot grasping task is presented. Speech and vision are combined to specify the command. In the manipulation stage, the robot moves by switching between different arm and hand modes (eg. approach, shape, grasp, release).

A simulation of grasping was presented, based on stored postures. Separate postures are used for the arm and the hand. The constraints, in order, were to avoid collisions, spatial accuracy and movement cost reduction. A search is performed through the postures and a goal posture is selected, and a via posture if necessary to avoid a collision. The predictions of this model were compared with experimental results. It was predicted and found experimentally that larger object sizes correspond to smaller aperture overshoots. A further prediction that larger objects cause the moment of maximum aperture to occur earlier was not seen experimentally. This model is limited in the sense that it is only a kinematic model but does manage to capture many of the properties of such movements.

Objects are modeled by primitives (boxes, sphere, cylinders and cones), and based on these primitives, one of two grasp preshapes is selected. As well as the type of preshape, the location and orientation of the wrist (6 parameters) and its orientation (2 parameters) are specified. A number (50 to 100) of possible preshapes are generated based on some simple rules defined by the type of object. These preshapes are tested by moving the hand to the object and closing the hand until contact occurs. The grasp is evaluated using a stability measure. Infeasible grasps (for example, because of an obstacle) are discarded. The best grasp can then be selected.

A simple rule is presented for predicting the relative durations of the opening and closing phases of the hand during prehension. They propose the duration of each phase is proportional to its amplitude (do and dc), i.e. To/Tc = do/dc. The relative time (To and Tc) to maximum aperture is thus determined by the ratio of opening and closing apertures. An experiment showed that 96% of timing variance is account for by this rule.

A Parameterized Self-Organizing Map neural network is used to learn the 20 joint angles of a hand only based on the locations of only the fingertips. This allows approximate reconstruction of the joint angles of the hand from only a small amount of information (the locations of the finger tips).

The role of the thumb in force production tasks in different grasp configurations was considered. When the thumb acts in parallel to the other fingers, it acted similarly to the other fingers (in that the force applied was less than if it applied force by itself). However, when it acted in opposition to the other fingers, the peak force was much larger than when it applied force by itself. They conclude that in some configurations (i.e., in parallel to the other fingers), the CNS treats the thumb as a fifth finger with respect to force deficit and enslaving, although the muscles used for the thumb do not have the relationships that exists between the other fingers. From this, they suggest that the magnitude of interaction between the fingers has a significant neural and not only biomechanical component.

A model for how infants may learn to grasp is presented. The model consists of several modules specialized for the task (a virtual finger layer, a hand position layer and a wrist rotation layer). The selected grasp is determined based on an input (the location of the target) according to a probability distribution. The feedback to the learning is based on a reward signal based on the grasp stability. The model successfully ``learns'' to grasp, similar to that of infants. Based on the model, they suggest that infants can acquire grasping rather than innately possessing it and that initially grasping is an open-loop process.

The coupling of the arm and finger movements during prehension tasks was tested by looking at the kinematics from an experiment with a double-step paradigm involving a task where the subject had to reach a grasp a dowel. The velocity profile of the wrist was bell-shaped, with 2 peaks seen in the perturbed trials. The aperture of the grip was seen to increase to a maximum, then decreased to close on the dowel. In perturbed cases, often two peaks were seen in the aperture profile. It was noted that the each peak aperture followed a wrist velocity peak, however they concluded from a statistical analysis that the two components are not systematically coordinated but rather time-coupled in some way.

An algorithm was devised for efficiently planning stable grasps (for a Barret hand) on undeformed superellipsoids. The set of possible grasps was parameterized using four parameters - two for the starting position of the palm, one for the roll and one for the spread of the fingers. For each superquadric, 3,600 grasps were generated that span the space. SVM regression was used to efficiently compute the grasp quality, and this quality was maximized for given shape parameters. The algorithm succeeded in producing stable grasps, however it was for a simple hand and class of shapes.

Grasp planning is considered here as the solution to an optimization process with certain constraints. The optimization condition is to minimize the difference between the finger force magnitudes. The conditions (such as force and moment equilibrium, contact forces) applied, are dependent on the type of grasping (normal, with controlled sliding, or regrasping).

Reach-grasp-lift movements were performed on virtual objects of identical size but different simulated mass and coefficient of friction with the floor. Haptic feedback was provided with a Phantom haptic feedback device. When the object was more stable (greater mass, or higher coefficient of friction, the contact force was greater. They suggest that this means the stability of the object is learned, and is used in planning the movements. Movement to more stable objects also showed different kinematic properties, in the form of larger hand apertures and velocities. If no haptic feedback is provided, the movements are similar to those of unstable objects, hence for movements involving stable objects, haptic feedback is needed to avoid unrealistic movement features.

The difference in grasping a real and a virtual ellipsoid was studied. Grasping the virtual object showed greater deceleration and variability - this is probably due to the lack of contact at the end of the the motion. Furthermore, the type of grasp selected was dependent on the amount of visual information given.

A mathematical analysis of force production in pushing a pivoting stick was performed to determine what is required to maintain static stability. The hand rotational and translation stiffness can be used to stabilize the stick. It is suggested that such a strategy is generally used by humans for force-production task. Such analysis can also be useful in tool design.

The properties of neurons in the dorsal premotor area F2 of macaque monkeys was studied during grasping tasks. The neurons were classified as purely motor, visually modulated or visuomotor, depending on whether they were affected object presentation, motor action or both. Some neurons showed preference for the type of grasp (e.g. side grip vs precision) and others for the size of the object.

The effect of changing the centre of mass and handedness was compared on the forces applied by the fingers during grasping tasks. For each finger the normal and tangential grip forces were measured. Similar patterns of forces were used despite the unpredictability of the centre of mass and with different hands. It was also noted that the normal forces exerted by the fingers are synchronized and usually in-phase or out-of-phase. This suggests that some sort of synergies are used for coordinating the fingers during grasping tasks.

A model of motion planning is presented based on stored postures. An initial goal posture is selected that satisfies certain constraints (eg it is close to the object, it doesn't collide with the object, travel costs are low), and postures close to this initially selected posture are also generated. The best of these is selected, and if collision will occur a via posture is also generated. The posture of the hand is first generated, then that for the arm. The movement is then executed. The predicted movements predicted well several features of such movements. It should be noted that the model is for movements in a plane, although they suggested how to extend it to a 3D model.

This paper formulates a description of dynamic manipulability, analogous to manipulability (velocity) ellipsoids, which gives the relationship between joint velocity and end effector acceleration. The effect of velocity is taken into account in the formulation.

Hand motion during reach to grasp of real, virtual and remembered targets were studied. By using PCA, it was found that two principal components can account for >75% of the variation. The first PC is made up of the extension and flexing of the fingers. The second PCA, which begin about 70% of the way into the movement accounted for the extension of the digits.

A series of experiments were performed looking at the accuracy of adjusting finger span to various objects. Different permutations were made - to size, shape, distance, orientation and finger configuration. None of these factors had a major effect on the accuracy, contrary to the findings of other studies. Whole hand movements to grasp a cube were also measured using the CyberGlove. Almost all the variance in these movements could be described using two principal components - the first remained fairly constant throughout the movement, and the second represented the bending of the fingers that varied throughout the movement. The small number of principal components needed to describe the movements however may be due to the specific task (grasping cubes).

It was found that the joint angles representing the posture of the hand while gripping imagined targets did not vary independently between objects. Rather, most of the variance could be described using a much smaller number of components. They suggested that this means that the hand posture is controlled with a few postural synergies.

It was found than when gripping concave and convex objects, the hand gradually mould to the shape. The posture of the hand discriminated between the shapes well before contact, although the discrimination was incomplete at the time of peak aperture. It is suggested that this is because this parameter is not fully specified until later in the movement.

The effect of object shape and visual feedback during grasping was studied by an experiment where subjects had to reach and grasp objects in different visual feedback conditions. They suggest from the results that at least two motor processes occur in grasping. The first is a preshaping of the hand (about 45% of the movement time), and the second is a slower grasp modulation to refine the grip to its final shape. Movement duration increaded with lack of visual feedback.

A comprehensive review is made of grasp synthesis algorithms for robotic grasping. Grasp properties are categorized according to grasp dexterity, equilibrium, stability and dynamic behaviour. Algorithms are suggested for synthesizing grasps with the desired properties.

This paper claims that their grasping model based on constraints on the end effector is just as effective as more complex models based on the posture of the arm and hand in predicting the main features of grasping movements. From this they say that postural constraints are not important in trajectory formation of reach to grasp movements.

A model of grasping is presented that rather than modeling the movement as two separate parts (transport and grip) models the movement on the entire movement of the thumb and fingers. This is based on the notion that it is the thumb and not the wrist that is transported during grasping. The movements are then planned using minimum jerk trajectories but with the assumption that the fingers and thumb approach the object perpendicularly. This model predicts several features observed in prehension movements, such as that the object size affects the maximum aperture but not the movement of the wrist.

Finger movements during typing were studied, using principal component analysis on each joint separately. This showed that only a few (2 to 4) principal components were needed to explain most of the variability of each finger. Cluster analysis was also used to test hierarchical relationships, and showed evidence of patterns between the joints, or synergies.

The number of synergies involved in some hand manipulation tasks is considered using Principal Component Analysis, based on the assumption that the first few principal components describe the main synergies involved in a task. They found that 6.5 Principal components are necessary to describe most of the variance for different manipulation tasks. For a task involving moving all the joints individually, they found that only 8.5 principal components are needed (due perhaps to biomechanical coupling). These results are higher than in simpler grasping studies, but do not show that the neural controller eliminates many of the synergies it has access to. Furthermore, different synergies were observed for different tasks and between subjects. Based on these results, they suggest a task-optimal control strategy (optimizing only parts of the movement related to the performance) gives a better explanation that simplifying the control.

A system is described for robot imitation of grasping movements. The system tracks the hands and fingers using a stereo camera. The tracking is performed based on Gabor jets, which measures the similarity of an image fragment to a template. The grasping is based on tracking the location of the index finger and thumb, and is implemented using a gripper.

A model was presented for planning reach-to-grasp movements. Three components were planned - hand position, grasp aperture and hand orientation. Each component is planned based on a difference vector between the current and desired position. Coordination between the components is achieved through a common (increasing) gating signal which ensures that the components end approximately simultaneously. An additional feature is introduced to the aperture control, called self-inhibition, which accounts for the tendency of the hand to return to a relaxed position. This model accounts for a nuber of features observed experimentally for such movements. Perturbations of the object are handled by altering the common gating signal. This model is implemented as a neural network.

The dextrous capabilities of the raccoon hand are presented. Although the raccoon does not have a truly opposable thumb, it is capable of dextrous manipulation. It achieves this by avoiding fingertip grasps, and instead using the palm or more commonly some other fixed surface (such as the ground) to constrain the object. The scissor grasp, grasping using abduction between the fingers is also sometimes used to constrain objects. They also grasp and regrasp an object a few times before being up, in this way it is believed that the dynamics of the object is learned and it is placed in a convenient orientation and location. They also make use of the two hands for grasping and manipulation. It is suggested that such techniques could be used in robotic hands which are less dextrous than the human hand.

A review is presented of the issues involved in multi-fingered grasping. He reviews the technicques involved in grasp stability, finger force distribution, and grasp compliance.

This paper studied the temporal organization during the activity of drinking from a bottle with a glass using two hands. It was suggested that the movement is organized such that synchronization will occur at critical times during the movement. This would be part of a top-down control mechanism for motor execution. Additionally they found that the temporal structure was common across different modalities (different forms of pantomime and with the real objects).

This review suggests that motor plans are represented in higher coordinate structures which then coordinate the necessary interactions at the lower executional levels. The context of the motor tasks influences the particular organization used (for example, compliant and unrestrained movements show different curvature).

Task-dependent preshapes are used as a way of simplifying robot grasp planning. A preshape is selected (by the user) depending on the task (e.g. precision, lateral or manipulation). The preshapes have parameters - these are fit such that the aperture is proportional to the expected grasp distance, but kept as small as possible. The finger movements are then generated to close on the object, using a proximal or distal strategy. This method de-emphasizes stability analysis, rather assumes that the selected strategies will lead to stable grasps.

A Hidden Markov Model is presented as a way of recognizing and emulating human gestures. This allows the invariants patterns in the movement to be found.

A system was developed for using a CyberGlove along with force sensors to describe the posture and force to apply for telerobotics. It was found that the grip size was primarily controlled by changes in the MCP angle, while the main force exertion is from the thumb and index fingers. It was suggested that these primary parameters can be used to specify the robot grasp.

A comparison was made of reach to grasp movements between normal subjects and subjects with Cerebellar damage. Those with the Cerebellar damage performed worse for isolated reach, and grasp, movements, although they did not worsen the parameters of these movements when they were combined, although other deficits were seen - more variation in the movements, and a separation between the reach and grasp components. This decomposition strategy is believed to be a default strategy for these subjects. It was concluded that the cerebellum is probably involved in the control of combined reach and grasp movements.

This paper explains a system of classifying manipulative hand movements (coordinated movements of the fingers to manipulate an object). The joints are assumed to be coordinated such that they are in-phase or anti-phase, and so can be expressed in terms of another joint. The movements are segmented on-line by detecting the ``folds'', and the 16-dimensional vector representing the joint angles with respect to the most active joint is classified using an ART network. High rates of recognition are achieved.

Manipulative hand movements are assumed to be coordinated, and hence straight lines are expected in phase plane (when two joint angles are plotted against each other). These straight lines are detected using the Hough transform and used as a basis for segmenting the movement.

The forces applied by the fingers in a task where the subject had to keep a handle vertical under differing load and torque conditions were studied. The moment required to keep the handle vertical was provided about 50% by normal forces and 50% by shear forces. The index and little finger torques were found to depend mainly on the torque, while the middle fingers depending on both the applied torque and the load. Additionally, antagonist movements were always seen, even when they are not mechanically necessary.

A Neural network model was used to explain the forces applied by the fingers in tasks requiring application of torque and force. Optimization of the finger forces could not explain the results seen, due to the effect of ``enslaving effects'', where a finger that is not required to produce a force is activated because of commands given to a different finger.

The synergies involved in a force and torque production task were studied. They defined a synergy as conjoint changes of finger forces and moments during multi-finger prehension tasks. Evidence was observed for use of synergies as a way of resolving the redundancy. For example, the adaptations made were of the synergy as a whole, rather than as a minor change.

It is claimed that, as is used in robotic control, humans perform superposition when performing prehension. They looked at grasping a handle with a prismatic grip with different applied torques. They observed no correlation between the forces needed to prevent the object from slipping and for maintaining the object orientation and hence concluded that they are controlled by separate commands. They also found that the finger force changes associated with the changing of one of the parameters did not depend on the other factor, and hence concluded that the two commands can be summed.

The synergies involved in a precision grip were reviewed. It was claimed that there are two independent commands, one to prevent slipping and one to maintain the rotational equilibrium, and superposition can be used to combine these commands. Due to the large space of forces that can be applied, these synergies can only identify a subspace of solutions, and some other mechanism needs to make fine adjustments to meet the task requirements.

The connectivity between two bodies is the number of independent parameters needed to describe the relative locations of the two bodies. It can be calculated from the mobility and redundancy of the system. The connectivity is used for classifying into three types of grasps - power grasps, constrained motion grasps and free motion grasps. These classifications can be used in grasp synthesis.

A set of canonical grasps are specified in terms of the location of the fingers and the object, and a graph is defined that specifies the possible transition between these canonical grasps. A manipulative movement is then programmed by finding a suitable path in the graph from the starting grasp to the final grasp. This method was tested experimentally using a three fingered robotic hand.

A quantitative measure of multi-fingered grasps is presented. It measures the capability of the grasp to hold the object under external disturbances. The measure is calculated from the set of contact wrenches. It allows grasp analysis and synthesis.

A method is presented for recognizing manipulative hand movements as part of a larger system for programming a robot by demonstration. The movements are segmented based on force sensors (to determine contact with edges). The grasps are further segmented between static and dynamic grasps. Dynamic grasps are classified according the taxonomy of Elliot and Connolly. Classification is performed based on joint angles measured with a data glove and a Support Vector Machine (SVM) classifier is used. High classification rates (around 90%) were achieved.

It was shown that sensimotor adaption where the scene was rotated 60 degrees is learnt between certain movement categories, namely tracking and pointing. The magnitude was larger from pointing to tracking. They suggest that adaptation is located in the brain before the divergence for different movement categories.

A simple sensory feedback scheme that operates in task space is described for controlling arm movements. This technique avoids the need to perform inverse dynamics or deal with excess degrees of freedom. The damping terms in the feedback equation can be selected to prevent self-motion and to cause the velocity profile to be roughly symmetrical and bell shaped. This model with appropriate selected parameters is simulated for a 4-joint arm model making movements in a horizontal plane.

Double step movements in grasping are investigated, where the target (a disc) sometimes changes size after the ISI time following the initial target presentation. In particular, they consider the aperture of the grip (the distance between the thumb and index finger). They consider whether the change in trajectory is due to cancellation, superposition or amendment. They conclude that neither of the three is a good description.

A biomechanical model of the index finger is presented that can predict the tendon extensions and forces based on the trajectories and applied forces. It is based on the combination of models of tendon extensions and forces. The unknown parameters are solved using a recursive Newton-Euler method under the additional constraint of minimizing muscle stress to solve the otherwise under-constrained problem.

The kinematics of point to points movements of the index finger moving in a plane were studied. It was found that the movements were not straight, and the path was dependent on the direction (i.e., a to b has a different path to b to a). From this they suggest that the trajectory plan must not be solely kinematic and must take into account mechanical properties. However, they did not control for starting or ending posture, which could also be the cause of different paths in the two directions.

A simple gesture recognition system is described based on the coordinated movements of landmarks

A model is presented for the control of a non-rigid object by the arm. As opposed to studies involving adaptation to perceptual or mechanical perturbations which are parametric perturbations, this is an example of a structural perturbation because new equations are required to describe this system rather than changing the parameters of an existing dynamic equation. The movement is of a virtual mass attached to the hand by a (virtual) spring. The boundary conditions of the movement are on the initial and final position, and that the hand and object velocity and acceleration should be zero at the start and end of the movement. This provides 10 independent boundary conditions (because of dynamic coupling), and a 9th order trajectory is used to describe the motion of the object and a different 9th order trajectory describes the hand motion. A 9th order polynomial is predicted by minimizing the mean squared crackle (5th derivative of position) of the object trajectory. For fast movements, this model predicts hand velocity profiles with 2 peaks, a direct contradiction of the predictions of the minimum jerk model. Such velocity profiles were experimentally observed. They note that this model, which they call the optimally smooth transport principle, is a descriptive rather than explanatory account of the movement. They suggest from anecdotal evidence that visual feedback is required to learn the task, because unlike movements involving the limbs, no proprioceptive feedback is available.

An experiment was performed with virtual targets, with single step and double step movements, where the target could change in size or position 300ms after the target appeared. The change in object size effected the kinematics of the grasp but not the transport component, while a change in object position changed the kinematics of both the grasp and transport components. The correction time was found to be distinctly different for the grasp and transport components. It was also noted that in cases of double-perturbation (change of position and size), these responses can not be thought of as a combination of two single-perturbed responses. They conclude that their data is consistent with a model for prehension based on two mutually coupled channels (for grasp and transport).

This paper classifies hand movements in terms of the types of synergies (simultaneous or sequential), as well as the patterns and digit groupings and use.

A route planning system is described that uses online control to determine the current state without an explicit world model or path plan. The route is planned using a dynamic model in terms of the angular acceleration. The goal acts like an attractor, and the obstacles like a repeller. Multiple objects can be simulated by linear combination. The routes predicted by the model were similar to those performed by humans in a Virtual Reality experiment.

A gyroscope was used to alter the dynamics of a hand movement. It was found that the hand path did not change, but that the configuration of the arm was altered. As the subjects learned the movements, the arm gradually returned to its normal configuration, implying that different forces were generated. The normalized peak of the kinetic energy did not increase with the learning - from this they suggested that kinematics and kinetics might be mutually optimized.

This set of experiments considered the effect of object affordances on grasp selection. An object affordance is a motor representation that causes particular types of interaction, such as the size of the section to be grasped or the object's weight. One theory states that only the relevant affordance for the task influences the grasp selection, while a second theory claims that the grasp selection will be influenced by all the affordances of the object. If this second theory is true, then the grasping of objects should be affected by object affordances which are not part of the current grasp. This was observed in a series of experiments, and based on this, the author suggests that objects has a single motor representation which is used in grasp planning and implementation.

A neurophysiologically plausible model is proposed for movement recognition. The model is also quantitative, allowing its predictions to be tested. It is based on two pathways, for form and for motion, analogous to the ventral and dorsal streams. Each pathway is a hierarchical model that begins with low level details - local orientation detectors for the form pathway and local motion detectors for the motion pathway. These low level details are combined hierarchically to give representations at different levels, until recognition can be performed. These levels are related to different brain areas. The model is capable of explaining the results of many existing studies.

A study of independence of finger movements found that when asked to move one finger, motion in the other fingers was also produced. This lack of individuation was the same for dominant and non-dominant hands, and less independence was seen when the frequency of cyclic movements was higher (for 3Hz compared to 2Hz). The unrequested motion may be due to passive mechanical connections, the organization of multi-tendonded finger muscles and from neural control.

Joint motion was measured with an ultrasound based motion analysis system. It was found that there is a linear relation between the proximal and distal interphalangeal joints, equal for flexion and extension. The ratio is 1 (PIP) to 0.76 (DIJ).

The relationship between muscle strength and noise was examined for different muscles in the arm during a torque matching experiment. The force was measured using a force transducer. The relationship between muscle strength and muscle noise for each muscle was calculated based on the maximum voluntary torque production. This was compared with the results of a muscle simulation, where the output of the muscles was the summed result of muscle twitches caused by a spike train with a Gaussian interspike interval distribution. The number of motor units and the spike train noise were varied. It was observed that as joint strength increases, the coefficient of variation decreases exponentially. The simulations were able to accurately model the data, from which they conclude that stronger muscles with more motor units have a lower coefficient of variation.

A review is made of strategies for dealing kinematic redundancy (or as he describes it, kinematic abundance). He considers methods based on relationships between the variables (such as using PCA) and those based on some form of minimization.

Four models were compared for direct and via-point pointing movements - minimum work, minimum angular jerk, minimum travel cost, and Donders' law. In terms of absolute error, Donders' law gave the best description of the data.

Different types of forward models are presented and it is noted that is necessary to first define what type of internal model (in terms of input space, output space and its structure) before evidence for or against the existence of such models can be considered.

In a study of adaptation to varying force fields during reaching movements, it was found that subjects were unable to adapt to a time-varying force field while they were able to adapt to a velocity-varying field. They speculate that the system that adapts movements to external forces cannot use a temporal representation.

A planning scheme for a 3 DOF robot finger is presented, based on the human finger. The key features is that the distal interphalangeal (DIP) joint and the proximal interphalangeal joint (PIP) are linearly related. It is compared to planning the movement in order to maximize a manipulability measure. They conclude that the requirement of interphalangeal coordination can produce natural trajectories.

The development of arm kinematics of infants is studied. Through the first two years, the path become nearly straight and the number of "movement units" decreases, and the movements become unimodal. Still, there are considerable differences between the movements of a 3 year old and an adult.

A series of experiments were performed to support the theory that the central nervous system uses probabilistic models during sensimotor learning. Using a virtual reality setup, subjects were believed to have learned the distribution of lateral shift which had a Gaussian distribution. This was tested by using different feedback conditions. The trajectories observed support such a model over a model where subjects estimate the average lateral shift, as well as a model where they learn a mapping from the partial feedback to an estimate of the shift. Subjects were also capable of learning more complicated distributions, such as a mixture of two Gaussians.

The strategies in a multiple finger force production task were compared between normal subjects and subjects with Down syndrome (DS). It was found that a simpler, sub-optimal strategy was used for controlling the force applied with the DS subjects, where they did not compensate for errors between the fingers. However, practice had a considerable effect on improving finger coordination with such tasks.

A finger force production task was examined. As seen previously, the variance in forces related to the task was much lower than the variance in forces unrelated to the task. Similar results were found between a discrete task (ramp force production) and an oscillation task. From this they concluded the synergy organization is the same between such tasks. It was also noted that the stabilization of force was only possible within a certain range of values for the force. It was suggested that this may be because error correction of the forces involves time delays that are too long to achieve stabilization.

This paper presents the Uncontrolled Manifold (UCM) hypothesis for analyzing variability in motor control. This hypothesis assumes that there is a subspace, the ``uncontrolled manifold'', for which the variance is not controlled, but only along ``essential'' directions that do not belong to the UCM. Hence high variability can be shown as long as it remains inthe UCM.

The extent of motion enslaving between the fingers is observed. Finger movements were restricted so that only the distal interphalangeal joints could move. Considerable enslaving was observed - the motion of one finger caused the slightly delayed movement of one or two slave fingers, with amplitudes for some fingers greater than 60% of their peak amplitude. The index finger was the most independent. Several possible explanations are given for this phenomenon.

The predictions of different types of control schemes in a pole balancing task were studied. This task was chosen because memorized motor commands cannot be used, rather, closed-loop visual feedback is needed. During some trials with a virtual pole, the visual feedback was blanked-out, but the subjects succeeded in maintaining to balance the pole as well as when they had feedback (up to a certain amount of time). From this, they suggest that there is a forward model in the control loop. They show that a delay uncompensated control model and a Smith predictor model can be eliminated as feasible control hypotheses.

A robot learning algorithm is developed based on spline fitting using via points. In an example using a simple game, via points are extracted from human demonstrations, in terms of Cartesian coordinates, and joint angles (from a 7 DOF arm). In reproducing the trajectory for the robot manipulator, the Cartesian coordinates must be exactly reproduced while the joint angles of the robot should be as close as possible to those of the human. The trajectory for the robot is generating by fitting splines to the via points.

Learning during adaption to a destabilizing perturbation of a knob turning was examined. After time, the subjects learned to move accurately under the perturbation and their overall kinematics and performance measures returned to close to what they were before the perturbation. They suggest that their observations support the hypothesis that subjects adapt by learning to make more accurate primary movements (i.e. without corrections) and the other measures (such as smoothness) can be explained as a secondary effect.

A method is presented for reconstructing the joint locations from markers placed in arbitrary locations on the limbs. The method is based on calculating the transformations from one limb to the next and finding a point that remains still between the transformations. A best-fit solution is found to take into account the noise in the system.

Subjects were able to learn two force fields that were applied in opposite directions during a centre out task. The force fields were velocity dependent, and one operated in a clockwise direction, the other counterclockwise. As opposed to other studies whether the opposing fields were presented in an alternating sequence, or in blocks, and subjects did not succeed in learning both models, they showed that using random presentation the subjects were able to adapt to both fields. The force fields were accompanied at the start of the movement by audio and visual cues to indicate which field would be presented.

A model for how infants may learn to grasp is presented. The model consists of several modules specialized for the task (a virtual finger layer, a hand position layer and a wrist rotation layer). The selected grasp is determined based on an input (the location of the target) according to a probability distribution. The feedback to the learning is based on a reward signal based on the grasp stability. The model successfully ``learns'' to grasp, similar to that of infants. Based on the model, they suggest that infants can acquire grasping rather than innately possessing it and that initially grasping is an open-loop process.

The performance during typing was measured when the index finger was anesthetized and in a control situation. The average kinematics remained the same - they suggest this is because typing is executed by an open-loop system. However, typing errors increased sevenfold, and much more variability was observed. Regression analysis showed that the endpoint variability was mostly predicted by variability of the starting position. They suggest that the starting position was poorly predicted due to the lack of tactile feedback (such feedback aids in accurately measuring the finger posture).

The movements of a 7 DOF human arm are recorded and used to teach a neural network about reaching movements. General movements can then reproduced by a virtual mannequin that have properties of human movements without them being explicitly specified.

The role of variability in motor behaviour as more than simply noise is examined. They describe how the strategy of first separating movements into deterministic and random components can lead to missing out important features of the movement. They suggest that the variability may even be more revealing than the invariants of the motions, and suggest tools for analyzing the variance.

A 3D model of the human index finger is presented that can be used for estimating the muscular forces involved in free finger movements.

A biomechanical model of the four fingers in the hand is described. Each finger is considered as an open chain of rigid bodies (the bones) connected at the joints. The movement of these chains is controlled by the muscles (of which 25 are considered) through the tendons. This model was used to predict the maximum voluntary grasping force for different sized cylinders.

A neural network is used to model the movements of a 3 DOF finger. The redundancy is dolved by assuming the PIP and DIP angles are equal. A minimum jerk velocity profile is assumed to produce smooth movements

An automatic method is presented for detecting biological movement from Johansson displays, based on maximizing the joint probability function of the position and velocity of body parts.

This paper compares the minimum crackle model for object motion with a minimum jerk hand constraint. The minimum crackle model, presented elsewhere, requires that the mean-squared-crackle of the object be minimized. However, they show that if the object is rigidly connected to the hand, then this does not reduce to the minimum hand jerk as would be expected. Also, if multi-mass objects (objects connected by springs) are considered for manipulation, then minimum crackle would be insufficient and higher order derivatives of position are required. Instead, they minimize the hand jerk, under the dynamic constraints of holding the object. Their predictions for multi-mass objects are much better at predicting such movements than an extension of the minimum crackle model. They note that this model is only presented for 1D (i.e., it does not predict the path), and does not require taking into account the inertial properties of the arm.

A review is made of using optimization principles in motor control, contrasting open loop and closed loop models. Open loop optimization models are presented as representing the average behaviour, while closed loop models allow integration of sensory information, i.e. optimal feedback controllers.

The constrained minimum-jerk model is presented in this paper. This model is based on the minimum-jerk model, but the jerk cost is minimized when the path is specified (rather than just the end or via points). They found experimentally in a range of movements that this model better predicts the velocity profile than the different versions of the 2/3 power law. They suggest that this model is applied over a small sliding windows (of approximately 1 second) rather than globally.

A model for planning movements is presented based on independently planning the geometrical and temporal components. Under different speed conditions, the the position-orientation hand paths were found to be similar. The kinematics on the movements were found to be dependent on the initial and final postures.

A gender classifier for point light display of a human walking on a treadmill is constructed. PCA is performed on the 3D locations of 15 markers and the first 4 Principal Components describe 98% of the variance. Each movement is then described as a trigonometric function in terms of the mean posture, the first 4 eigenpostures (principal components), the fundamental frequency and phase shifts of the eigenpostures. The eigenpostures were similar across subjects. PCA is performed again on these descriptions, and a linear classifier used on this representation. 90% correct classification is achieved (compared to 76% by human observers, although the humans only got a 2D representation). Size was a good cue, as well as dynamic information.

The UCM approach was used for studying the coordination involved in pointing movements. The aim was to partition the variance across multiple repetitions into joint angle variance (10 DOF) that affects the goal, in this case, the spatial position of the hand, (NGEV) and that which does not affect the goal (GEV). It was found that the GEV was significantly greater than the NGEV, i.e. that subjects use a range of goal equivalent strategies to achieve the goal

The variability of endpoint locations in a centre-out movement task with an unseen hand in a plane was examined. They assumed that the noise is added at the level of the motor commands. Based on a model of the kinematics and dynamics of the hand which relates the motor commands to the final movement, they predicted the endpoint variation caused by the noise. This accounted for much of the variation observed in the movements. A simpler model of noise in planning does not explain well the variation. They suggest that the noise consists of a combination of signal-dependent and signal-independent noise.

A description is presented of the relationship of tool use and human evolution, and how tools are used to amplify force, and factors in tool design.

The importance of tool use in human evolution is presented. It is believed that complex society evolved from the use of tools, and that tool use was the both the cause and effect of human development.

Different computational models are reviewed for four areas of motor control - motor planning, motor prediction, state estimation and motor learning.

Finger tapping where one hand has a constant phase shift to the other was modeled using two coupled oscillators. The predictions agreed with the experimental findings, that is, that the stable states are when there is 0 or 0.5 phase shift. These states showed the smallest errors, and when starting close to one of these phase shifts, the movements tended to a stable point.

A dynamic model is constructed for finger movements (not for the thumb) based on measured joint angles (and velocities and acceleration) and measured and assumed quantities (link lengths, masses, inertial properties, etc). The finger joint moments were calculated when there is no external load. The model was able to accurately describe the velocity.

A model is presented for the imitation of motor skills. The movements are recognized by a visual system which identifies the direction and orientation of movement of each of the limbs to be copied. A hierarchical neural network, roughly based on the structure of the brain, is used to learn the movements. The learned movements were then applied on a computer simulated humanoid avatar. The learned movements were reasonably close to the presented movements.

Using fMRI, areas of the brain are found that are active both during a finger movement and the observation of someone else performing the movement. It is proposed that this area is involved in the process of imitation.

Experiments were performed where subjects watched video sequences of eye, hand and movements, sometimes with the intention to imitate and other times just to watch, while the fixation of the eye was measured. It was found that the eye mostly fixates on the end effector and not the other parts of the arm, regardless of the intent (to imitate or not). They suggested that this means people can fill in the details of the rest of the arm and so use learned models of motion, perhaps by building up motion primitives.

The theory and implementation of Laplacian eigenmaps is presented. Laplacian eigenmaps map high dimensional data that is assumed to lie on a low dimensional manifold into a lower dimension. Laplacian eigenmaps preserve the local information.

A model is presented for learning a centre-out task for a 2DOF limb. Using neural networks and feedback, the model learns an internal model of the dynamics of the arm and of the task. During learning, it minimizes a cost function that represents changing states. Although no trajectory is given, it ``learns'' to plan roughly straight movements to the targets.

Presents algebraic and geometric techniques for least-squares fitting of circles and ellipses. The two techinques give different results.

A technique for smoothing trajectories is presented based on smoothing in phase space, that is, simulateously smoothing over position and velocity. The smoother minimizes the error against both position and velocity while penalizing the roughness, based on the acceleration.

Different methods for representing translations and rotations, such as exponential matrices, quaternions and rotation vectors, are reviewed. An example of some of these moethods is given for a robot.

The Isomap method for dimensionality reduction is presented. Its advantage over PCA or multidimensional scaling is that it can handle nonlinear relationships between the variables, and find a low-dimensional non-linear manifold upon which the data sit.

An extension is made to Hidden Markov Models to include extraction of a parameter (e.g. some spatial feature of a gesture). This is instead of other methods where first a gesture is recognized and then the parameter extracted using an ad hoc method. The parameter is introduced to the HMM formulation by making the output density a function of the gesture parameter vector. A modified EM algorithm is described to find the parameter vector which maximizes the probability of the observation sequence. A nonlinear form is also described which allows the output density to not be linearly dependent on the parameter vector.

The difference in grasping an object with different visual feedback was studied. The subjects were shown, before the movement, either the real object, a virtual computer rendered object or a symbolic presentation (using a mirror setup). The visual information was removed at the initiation of the movement. Haptic feedback was provided (using a real object). Different kinematic properties were compared (e.g. preshape aperture, grasp onset latency, movement velocity), and no significant difference was seen between grasping real and virtual objects (as opposed to pantomiming behaviour found in other studies).

An experiment was performed to examine the dispersion of endpoints when the target but not the hand is visible throughout the movement (the target was a 3D rendition of the cube). They found that the errors were primarily in judging the endpoint. It was suggested from the findings that final position rather than intended displacements guide such movements.

The adaption of humans during goal directed movements when the visual feedback is perturbed was tested. The visual feedback was perturbed by either a translation, a scaling or a rotation in Cartesian coordinates. The subjects had to align a real cube on a stick with a displayed cube (they could not see their real hand). Adaption was not complete - 40% for translation, 20% to scaling and 10% to rotation. These differences were explained by comparing the ease in which these transformations can be generalized within egocentric frames of reference. They concluded that adaption to perturbations is performed by more than one mechanism.

In this article, the authors question whether the change in peak grip aperture observed when grasping the Müller-Lyer illusion means that people are using the change in size information for planning the grasp. However, if this is the case, then they should misjudge the size of the object, and there should be visible differences in the velocity profiles. As these differences are not seen, they claim that there is no evidence that the perceived size difference is affecting the grasp planning, and the larger aperture may be because the grasp is planned more carefully (because part of the illusion may be considered as an obstacle).

An experiment was performed with schizophrenic and regular subjects where they had to perform a movement and simultaneous view a movement that could be their movement or that of someone else. Then then had to judge whether the movement was theirs or not. The schizophrenic cases made many more errors in judging correctly. It is noted that there are different but overlapping parts of the brain for observing and for execution. They suggest that in schizophrenic patients, intending and observing a movement would produce indistinguishable patters, which would not be the case for regular subjects.

The question of whether the processes underlying perception and action share a common representational framework is explored. They conclude that there is not yet conclusive evidence for a clear neurophysiological substrate supporting a common coding.

A test was mode of Milner and Goodale's two visual system model (conscious perception and perception for action). Stimuli were found, using variants of the M\:uller-Lyer illusion, that affect only perception or action, but the findings were not consistent with the theory. Thus they suggest that ventral and dorsal give different information but the use of the two is task dependent.

The neural basis for intention is reviewed. Rather than intention being the start of an action, it has been found to be the direct result of pre-movement brain activity in the frontal and parietal motor areas. Intention allows us how a sense of agency as it can be determined whether an external event was related to a self generated event.

The perception of shapes based on proprioceptive feedback and not vision was studied using a robot manipulator to have subjects trace out shapes, and then to reproduce them. The shapes were reproduced more symmetrical or regularly proportioned than they really were. Based on different forms of reproduction, they suggest that the errors are due to distortions in the haptic synthesis rather than motor factors.

A review is presented of the research about human perception of 3D shape. While judgments of 3D shape are often systematically distorted, these representations are constrained. Furthermore, it appears that there is a consistent representation based on salient features such as occlusion or high curvature.

A hierarchical system of primitives is suggested for generating arm movements. The system is made up of fixed basic primitives, and another layer of adaptive primitives that exert control through the basic primitives. The adaptive primitives ``learn'' the movement.

The use of schemas is presented for planning reaching and grasping movements. There are schemas for different parts of the motion - that are constructed hierarchically. For example, there is a hand preshape schema. Also suggested for a cup grasping task is a virtual finger schema, where several fingers may be considered to act as one finger.

Primitives are used to learn how to play air hockey, which is then implemented with virtual agents and humanoid robots. The primitives are based on the different types of possible actions in an air hockey game (e.g. left hit, block, prepare).

Structures in the spinal cord of a frog have been found that produce a certain contraction in a group of muscles, and the result of activating different structures is the vectorial summation of these force outputs.

A method is presented for finding both the best set of synergies, and their offset and scaling coefficients to match a data set. Each time-varying synergy consists of a number of vectors (equal to the number of DOF) describing the movement from some class, e.g. Gaussians. A novel movement is constructed from the superposition of several synergies, which may be time-shifted. A non-negative matrix factorization algorithm is used. An example is presented for the activation of 5 muscles in the leg of a frog.

This is an expansion of the NIPS paper. Synergies consisting of Gaussian profiles for several muscles are automatically determined, along with their time shift and amplitude scaling for a large collection of frog EMGs. The technique is novel in that the synergies are automatically determined from the data, and they can be superimposed, with time shifts. Three time-varying synergies were sufficient for describing a range of different movements. Different synergies activations were observed for different types of movements.

This extends on their previous studies of synergies, by allowing a muscle pattern to be reconstructed from multiple instances of a synergy.

Based on the correlations observed between the movements of the joints during a set of simultaneous hand movements, a classification technique is developed the can accurately classify simultanesous hand movements.

Data is collected from humans arm movements and segmented (using a measure of angular velocity). A 4 DOF arm is considered, with each segment scaled to be 100 time units. Hence they have a 400DOF system. The primitives were generated using principal component analysis. These primitives were used to reconstruct the training movements and to generate novel movements.

A multifingered robot was considered as two virtual fingers, each with 3 DOF. The primitives were perceptual goals (eg move along a certain axis, rotate about a certain axis), defined by the programmer, which the robot then learns how to do by experimentation using the evolution strategy. These primitives can then be combined for use in teleoperation, although they only suggested how to do this manually.

Precision grasp-like movements with the thumb and index finger were performed, and the joint angles, velocities and acceleration were measured with the CyberGlove. Inverse dynamics were then performed to estimate the joint torques, on which they performed PCA to joint torque synergies. Although the Principal Components for torque described less variance that those for joint angles, under different conditions (faster or slower velocity), the joint torques were observed to scale linearly with the velocity.

A method is presented for generating trajectories using dynamic primitives for each DOF. The primitives are in the form of nonlinear dynamic systems. In the examples presented, 20 primitives per DOF were used with fixed parameters. In learning from demonstration, these primitives are fit using an incremental least squares regression. These primitives have the advantage that there is no explicit time dependency, they can deal with perturbations during the movement and each primitive shows bell-shaped velocity profiles. This algorithm generated the joint angles, which would then require an inverse dynamics controller to give the final motor command.

Movements are automatically identified as belonging to a certain primitive. This is done by finding the best match of certain movement features (zeros of velocities in some key coordinates) for a movement segment. The difference from this segment to the reference trajectory is expressed as a spatio-temporal transformation. Novel movements can be created by the linear combination of several of these spatio-temporal transformations. This was used for generating realistic looking karate movements based on some examples. It can also be used for exaggeration of movements and analysis of movement styles.

For recognizing an alphabet for fingerspelling, PCA is compared to using a subset of the joint angles. It was found that using a subset of joint angles was superior to using a similar sized PCA weighting vector in transmitting information about the pose. They suggest that hence synergies are not used as a primary control strategy for this task, and that recognition can be performed more easily and with less measured angles using this technique rather than PCA.

A system is presented for recognizing gestures based on hand (rather than finger) movements. The location of the hand is measured, and gestures are started and finished using a pinch sensor. Each gesture is made up or a number of strokes (ie up, down, clockwise from left, etc), and each stroke is recognized by a Hidden Markov Model (HMM), and a gesture is recognized by a combination of the strokes.

This work studies the temporal synergies observed in EMG when performing ASL finger spelling. They studied 27 transitions between letters, and performed PCA on these transitions (on 8 measured EMG signals). A small number of synergies (4) can describe most (80%) of the variance. The main synergy caused extension in the finger extension, occurring early in the transition. Later, the other synergies were responsible for thumb movements and later the finger flexion.

Using Single Value Decomposition (SVD), reach-to-grasp movements to various objects were analyzed. Similar results to those for static postures using PCA were found, in that most of the variation could be described by the first (97.3%) and second (1.9%) eigenposture. SVD has the advantage over PCA that it allows temporal interpretation rather than looking at the data statically.

A review of the use of primitives as structures that link the visual and motor system is presented. Different types of primitives, as well as methods of automatically learning them from data are suggested.

Three different control methods are used to make a simulated torso dance the Macarena

A primitive description was used for controlling a robotic dextrous hand. The primitives are defined as task specifications, and define, for each fingertip, and for each axis (x,y,z) whether force or position control should be used and the appropriate magnitude. To simplify two finger force application, task partitioning was used, that is, one finger (or virtual finger) remains rigid in the direction of the grasp while the opposing finger modulated the grasp. The elementary tasks were combined using a finite state machine to perform more complex tasks such as removing a child proof bottle top.

A methods is described for programming robots based on describing the constraints on its movement in terms of which axes it is free to rotate and translate about.

A set of primitives is defined for robot manipulation tasks. The primitives are described in terms of the the relative number of degrees of freedom (that is about which axes translation and/or rotation are possible). Force primitives involve changing the degree of freedom state, for example, aligning an object on a surface from a non-contact start. Such force primitives are defined in terms of an algorithm involving the requirement movement and the sensory feedback. Visual servoing primitives implement movement under certain constraints using visual feedback from a camera.

This paper presents an overview of the problem of inverse dynamics, and considers several solutions, explaining in detail the use of spinal force fields as primitives of motion and techniques for their combination to perform movements.

It has been suggested that complicated arm movements can be built up from the linear superposition of ``primitives'', in the form of non-linear, time varying force fields. A possible form for these force fields is presented based on pulse and step signals, and the predictions of the model are compared with experimental data.

This paper specifies the problem of inverse dynamics, and explains the equilibrium point theory. The problem of inverse dynamics can be solve through the superposition of force fields as primitives of motion. The stability of these force fields means that the solution corresponds to the virtual trajectories of the equilibrium point theory.

Subjects were required to turn a knob to certain illuminated LED targets. These movements consisted of a large primary movement, and sometimes an additional corrective movement. This additional movements took place either before the primary movement ended or after it ended. The different movements were identified by looking at inflections and zero crossings of the the jerk and snap of the movement.

Movements consisting of turning a knob to a certain location were modeled in terms of a primary movements and sometimes by the superposition of an additional movement, that could take place before or after the primary movement had finished. The movements were analyzed in terms of the angle of the knob being turned rather than the joint angles of the hand. The superposition was performed in terms of velocity.

Subjects are required to imitate arm motions, and the results of the imitation are compared in terms of the joint angles by various methods. They found that simultaneous rehearsal while watching impaired performance, and that performance did not improve with repeated trials.

A method for having a humanoid robot catch balls thrown at it is presented. The location of the ball is tracked by two external cameras, and a point of collision is decided upon, which is then moved to with a point-to-point movement, and a smaller follow-through is added. These movements are generated from motion primitives, in Cartesian space, in the form of programmable pattern generators (PPGs), which produce smooth trajectories. Inverse kinematics is used to transform this into joint coordinates to effect the movement. This algorithm is reasonably successful in catching the ball.

The learning of touch typing by novices enrolled in a touch typing class was studied. The movements of the hand and fingers at various stages were recorded using the CyberGlove and Fastrak. Stereotypical hand and finger patterns were developed early on. Coarticulation was seen between the hands during the learning. This increased movement overlap account for most of the decrease in movement times.

Subjects needed to copy a smooth movement, then in subsequent iterations they copied an average version of their last two movements. The movements then converged generally to movements that can be described, using principle component analysis, using a small number of parameters.

This paper suggests a pattern generator that is capable of generating both rhythmic and discrete movements. The movement is defined by a series of differential equations that ensure the desirable properties (for example smooth bell shaped velocity profiles in the discrete case).

This paper suggests a model of learning for humanoid robots based on the combination of movement primitives.

A model is presented for learning a centre-out task for a 2DOF limb. Using neural networks and feedback, the model learns an internal model of the dynamics of the arm and of the task. During learning, it minimizes a cost function that represents changing states. Although no trajectory is given, it ``learns'' to plan roughly straight movements to the targets.

A set of primitives is defined for a dextrous robotic hand as a sequence of joint angle changes representing some functional motion. Examples of primitives are closing the hand, pinching, rotating. The primitives can be added, subtracted and scaled to produce a wide variety of movements. The planned primitives are compared to the actual trajectories and adjusted according to a learning protocol.

An internal model was used to model the transformation of trajectories into muscle forces. It was modeled as a map from the velocity to the approximate force. It was assumed that the internal model learns the appropriate map based on the experimental error. The primitives were set to have broad Gaussian shape. The predictions of this model for data outside the training set were good.

An analysis of different manipulation tasks involving real objects using methods such as principal components analysis found that the number of degrees of freedom involved in the task were between 9 and 12, much greater than those found in previous studies (2-4). They explained that the previously assumed small number of synergies was due to the specific task.

The number of synergies involved in some hand manipulation tasks is considered using Principal Component Analysis, based on the assumption that the first few principal components describe the main synergies involved in a task. They found that 6.5 Principal components are necessary to describe most of the variance for different manipulation tasks. For a task involving moving all the joints individually, they found that only 8.5 principal components are needed (due perhaps to biomechanical coupling). These results are higher than in simpler grasping studies, but do not show that the neural controller eliminates many of the synergies it has access to. Furthermore, different synergies were observed for different tasks and between subjects. Based on these results, they suggest a task-optimal control strategy (optimizing only parts of the movement related to the performance) gives a better explanation that simplifying the control.

A technique is described for constructing hierarchical (two-level) approximately optimal controllers for movement. The high level parameters consist of parameters that are task-related (e.g., performance index) but also some state-dependent information. The low-level, which receives feedback from the movement and proprioception, is required to produce the parameters specified by the high level controller. The hierarchical approach allows optimizing the high-level controller without problems of high dimensionality.

Ten degrees of freedom pointing movements (three describing scapular motion in addition to the seven ``regular'' joint angles) were analyzed to see how the nervous system manages motor abundance (the extra degrees of freedom). Movement synergies were studied using PCA - more than 90% of the variance can be accounted for by one principal component. Also, the UCM (unconstrained manifold) approach was used. This compares variability that does not change important performance variables (GEV - goal equivalent variance) and variability that does change these variables (NGEV - non-goal equivalent variance). For most of the movement path, GEV was significantly higher that NGEV. They suggest that the different patters of joint coordination are not because of noise but rather represent equivalent solutions for stabilizing important performance variables.

The typing patterns of professional typists were studied. It was found that at normal typing speed, although the duration for typing specific words fluctuated, the temporal structure was invariant. This invariance was not seen for repeated trigrams. It is proposed that this invariance is due to word-specific templates.

The EMG during grasping objects and performing ASL sign language were measured and compared to the first few principal components. The first three EMG synergies (principal components) could account for 80% of the variance in the static EMG levels, but they did not show a simple correspondence to the joint angle principal components.

The effect of the Brentano illusion on pointing movements was studied. All pointing movements along the shaft were influenced by the illusion. When the hand or the target was unseen, the effect was greater (because on-line correction could not take place). For movements perpendicular to the line a very small (probably perceptual) effect was observed.

By using an experiment where subjects where required to orient a cylinder to different orientations and considering the errors, the frames of reference used for the movement were considered. The results suggested that such reaching and grasping movements are a combination of two frames of reference - one fixed in space and one fixed to the arm.

This reviews considers the representations of actions, particularly looking at the similarity between actions performed and those witnessed or imagined. This has been observed in similar activation of pathways in the brain. This review also looks at the different levels of processing, and notes that the unconscious level is distinct from the conscious level and has faster access to stimuli and is able to react faster and automatically.

This reviews looks at the effects of illusions in grasping tasks. Different experiments are considered from the viewpoint that an illusion will only affect a task if that spatial attribute is used in the task. For example, an illusion that affects the perception of the length of an object but not the end point positions will not have an effect on the perception or action if the length is not used but rather the end points. The differences seen in perception and action are explained by the different aspects used.

An experiment was performed to examine the dispersion of endpoints when the target but not the hand is visible throughout the movement (the target was a 3D rendition of the cube). They found that the errors were primarily in judging the endpoint. It was suggested from the findings that final position rather than intended displacements guide such movements.

The adaption of humans during goal directed movements when the visual feedback is perturbed was tested. The visual feedback was perturbed by either a translation, a scaling or a rotation in Cartesian coordinates. The subjects had to align a real cube on a stick with a displayed cube (they could not see their real hand). Adaption was not complete - 40% for translation, 20% to scaling and 10% to rotation. These differences were explained by comparing the ease in which these transformations can be generalized within egocentric frames of reference. They concluded that adaption to perturbations is performed by more than one mechanism.

A simple sensory feedback scheme that operates in task space is described for controlling arm movements. This technique avoids the need to perform inverse dynamics or deal with excess degrees of freedom. The damping terms in the feedback equation can be selected to prevent self-motion and to cause the velocity profile to be roughly symmetrical and bell shaped. This model with appropriate selected parameters is simulated for a 4-joint arm model making movements in a horizontal plane.

In this work, a cybernetic hand, called the ``cyberglove'' is presented. The cyberglove has 6 actuators (motors), controlling the four fingers independently and the thumb. Each of the four fingers has three joints which are controlled by one ``tendon''. The thumb is controlled by two motors. The hand is able to perform opposition with the thumb, and can perform lateral pinch, cylindrical, spherical and tripod grasps. The high level control (i.e., selection of which grasp and amount of force) will eventually be based on EEG / EMG signals. The low level control is responsible for actuating the desired force. Some sensory feedback is also collected.

The use of a wrist and an active hand are suggested as a way for giving better control for robot manipulators. Gross motions are performed by the arm, while fine motions by the wrist and hand. The hand may or may not be synchronized with the arm, dependent on the task being performed. Different grip are compared and rules are suggested for selecting grips.

A control strategy for a robotic arm is suggested that minimizes impedance, in order to make a robot that can handle unknown objects or objects in its path with a minimum contact force to avoid damaging the arm or the object.

A method for automatic robot operation from human demonstration is described. It is based on recognizing the configuration of objects before and after a task and determining the necessary transformation.

In the first part of this book, Mason analyses different types of contacts, using the notation of screws, twists and wrenches. He uses this to define which hand grips are stable. The grip transform is introduced as a way of transforming forces applied by the fingers to the force applied to the object. Stiffness control as a way of controlling the hand is also presented, as well as the design of a robotic hand (the Stanford/JPL hand). The second part of the book by Salisbury looks at the mechanics of grasping and pushing.

Behavior-based robotics is based on a set of behaviors, each of which achieves/maintains a specific goal (eg avoid obstacles, go home). This paper reviews the design and use of such behaviors for robotic control.

A review of issues in designing robotic hands is presented as well as different measures of hand design.

An algorithm for motion planning for a 3DOF prosthetic finger is described. The finger moves according to a minimum jerk trajectory (i.e. a straight line). It is assumed that there is a constant relationship between the two inter-phalangal joints. The relationship is found by selecting the value of the ratio such that it minimizes the maximum jerk of the three joints. The joint angle trajectories are found by approximating the movement with splines.

A set of primitives is defined for a dextrous robotic hand as a sequence of joint angle changes representing some functional motion. Examples of primitives are closing the hand, pinching, rotating. The primitives can be added, subtracted and scaled to produce a wide variety of movements. The planned primitives are compared to the actual trajectories and adjusted according to a learning protocol.

A real-time inverse kinematics method for humanoid robots is presented.

A system is described for robot imitation of grasping movements. The system tracks the hands and fingers using a stereo camera. The tracking is performed based on Gabor jets, which measures the similarity of an image fragment to a template. The grasping is based on tracking the location of the index finger and thumb, and is implemented using a gripper.

A formalism for performance measures for robot manipulators is presented. A configuration space is selected (such as the joint space of the arm) and a metric on it, a task space (such as the 3D position and orientation) and a metric on in, and a mapping between them (such as the forward kinematics). Based on these, several measures are defined - of the nonlinearity of the motion, of the relative directional performance, of the relative average performance and of the performance anisotropy. These measures can be applied for example on a Euclidean metric and an inertia metric. Examples on a non-redundant two link manipulator and a redundant three link manipulator are given.

The change in elbow joint stiffness as a function of the frequency was examined for cyclic vertical movements. It was found that the elbow joint stiffness showed a quadratic trend, with a minimum peak at a frequency close to the natural frequency.

The requirement of force-closure for grasp controllability was questioned and it was shown that grasps that are not force-closure can be controllable by utilizing gravity. The requirements for a controllable grasp are presented. A grasp quality measure is defined based on the grasp controllability. This can differentiate between kinematically identical grasps which have difference stability due to gravity, which wouldn't be differentiated by a measure based on the grasp Jacobian.

A method is presented for measuring endpoint stiffness during a movement. They predicted the trajectory of the movement based on previous trials. This allows the manipulator to first match the trajectory, and apply a perturbation made up of the predicted trajectory plus the perturbation (a constant displacement). This allows accurate prediction of arm stiffness (from which joint stiffness can be inferred). This method also allows measuring stiffness during adaption.

This study showed that the Central Nervous System can control endpoint impedance of the hand without modifying the posture or applied force. This was shown by having subjects move in a (unstable) divergent field. Subjects were required to move their arm vertically, if they shifted away from a purely vertical line, the force would push them further to the side. Increasing the stiffness horizontally (in the direction of the stiffness) means that the divergent field would have less effect. The subjects managed to increase their stiffness without relying on changing their posture (this was not possible due to the experimental setup).

The stiffness of different types of pen grasps is measured using a custom-built device. The Cartesian stiffness is presented as stiffness ellipsoids.

It is shown that the conventional mapping between endpoint Cartesian stiffness and finger causes discrepancy in the work profiles, because the mapping is not conservative. They introduce a new mapping, which they call the Conservative Congruence Transformation, which includes a term which relates the change of geometry (reflected by the change in the Jacobian) and applied force. This allows the physical properties of the stiffness matrices to be preserved in different representations.

An impedance control scheme is used for a robot manipulator for a task of wiping a surface. The end effector stiffness and inertia are controlled, and converted to the actuator torques using the Jacobian. The necessary impedance values for uneven surfaces was learned using an Associative Search Network (ASN).

A formalism is described for compliant motion, as an extension of Mason's hybrid control. It consists of selection of the task frame relative to the end effector, constraints on the force, velocity or tracking (detection of errors based on forces or velocities) in 6 dimensions in the task frame, additional task frame or end effector motion constraints, feedforward velocity constraints and task termination conditions.

A framework for implementing compliant robot motion is presented. The system receives as input the constraints as described in a previous work. It is based on a multidimensional position control loop embedded in a multidimensional force control loop.

Two methods are presented for modeling flexural joints, that is, two rigid bodies coupled by an elastic body. One method is based on twists, the second on dual quaternions. Both methods are frame invariant, and can be used to analyze displacements and the strain energy. An example was given of simulating a complex, many body flexural mechanism.

An impedance controller is described, that is, the torque necessary for a task can be computed as a function of the impedance parameters. These parameters are described as the translational and rotation stiffness and the damping in a chosen frame of reference. An example is given for an assembly task.

The causes of the observed stiffness in the human arm for movements in a plane with a manipulandum were examined. Previous studies showed that the major axes of the stiffness ellipses in different locations in the workspace were nearly co-aligned with the radial axis of a polar coordinate system. Simulations showed that the joint stiffness must vary throughout the workspace in order to describe this observation. They concluded that although the redundant number of arm muscles allows the central nervous system to select stiffness appropriate for a task, the constancy of the stiffness orientation observed does not support the notion that this redundancy is used.

Arm movements in a plane were studied in unloaded cases and when moving with an elastic load. After a few trials, the loaded trajectories converge towards the straight hand paths seen in the unloaded case. A ``summation'' model was proposed to deal with these cases, whereby the impedance parameters of the arm are found by summing the old motor plan and another ``unit of action'' responsible for dealing with the change in external load. It was claimed that the minimum jerk model is a better descriptor for such movements than the minimum torque change model due to the similar hand paths observed despite the different torque requirements.

The change in arm endpoint stiffness was compared after adaptation to two different types of force fields, a stable, velocity-dependent force field, and an unstable divergent position-dependent field. The joint stiffness changed in both cases. In the case of the velocity-dependent field, the eventual changes could be explained as a byproduct of the development of the inverse model, that is, different torques are required which affects the joint stiffness and hence the endpoint stiffness. In the case of the divergent field, the stiffness appeared to be directly controlled and tuned to the instability of the force field.

Based on experiments on learning in a velocity-dependent force field, and a position-dependent divergent force field, a model was proposed for how the Central Nervous system adapts to these changes. The subjects learned to compensate for both types of force fields. In both cases, muscle activation increased initially, followed by a gradual reduction. This activation could be related to increased stiffness. In the velocity dependent field, an inverse dynamics model which is generated to match the field, allows the trajectory to return to straight lines, and then the muscle activation is reduced to minimize metabolic activity. In the case of the divergent field, after it is observed that the dynamics of the field can not be learned, the muscle activity is reduced to the minimal level while maintaining the necessary stiffness to produce successful trajectories.

As was previously observed, arm stiffness was selectively altered in response to a divergent force field. This was modified independently of force and torque. Additionally, it was found that a constant net level of stability is maintained, after taking into account the instability of the environment. It is suggested that this causes the metabolic cost of the movements to be minimal.

The inter-finger connection matrix (IFM) quantifies the interaction between the fingers during force production tasks. This study compared the differences in IFM between different subjects. When the matrix is normalized, it is possible to use multi-dimensional scaling to identify two interpretable dimensions - the force sharing pattern between the fingers, and the contribution of the enslaved fingers to the total force.

To study the validity of the equilibrium point trajectory, human arm stiffness during multijoint arm movements was studied using a specially constructed manipulandum. It was found that the predicted equilibrium point trajectory using the calculated stiffness differed from the actual trajectory, which suggests that a more complicated model than the equilibrium point trajectory is needed.

The stiffness of multijoint arm movements in a plane was estimated using a specially designed manipulandum. The stiffness ellipses during movements were observed to have a much larger size (about 7 times) than during a corresponding relaxed posture. The equilibrium-point trajectories predicted based on the stiffness were quite different from actual hand trajectories. They suggest that at early stages of movement learning, high stiffness could be used to avoid disturbances, which may correspond to equilibrium point control. However, with the acquisition of an internal model that predicts the forces, the stiffness would be reduced to produce a less fatiguing movement that does not correspond to an equilibrium point trajectory, as were observed here experimentally.

It was shown that it is possible to alter the stiffness properties of the arm by changing the joint stiffness in addition to the well known method of changing arm posture. The variation was observed in a change of size, orientation and shape of the stiffness ellipse. This supports the idea that the stiffness can be changed for a particular manipulation task.

The forces applied by each finger in force production tasks was calculated by considering all the possible interactions between the fingers taking part in the task, both those that are applying force because of a neural signal and those due to enslaving. For example, if four fingers are applying force, then the force applied by one finger will be be due to first order indices (due to each finger), second order (due to two fingers), and so on up to fourth order. This leads to non-linear behaviour, and can be used to model experimental findings.

The impedance properties of the MCP joint of the index finger were studied by displacing the fingertip causing extension or abduction, and measuring the displacement and restoring force. The impedance was considered in terms of a static component (stiffness) and dynamic components (damping and mass properties). The first 20ms were considered for modeling the movement, before reflex or voluntary contractions take place. Each property was modeled by a single number. The impedance parameters were observed to vary with the external force.

A novel method of representing surfaces for virtual reality is presented. They use a series of rods in a grid to present the appropriate forces, with a surface attached to the outside.

This paper describes how the multi-joint nature of human limbs can be used advantageously to control the stiffness and inertial properties of the endpoint, which would not be possible with a single joint. He suggests that the excess degrees of freedom may offer a solution to the problem of controlling interactive behaviour rather than causing a problem. He also presents the idea of planning movements using a ``virtual trajectory'' consisting of time-varying equilibrium postures that allows the movement to be planned without requiring the computation of inverse kinematics.

This paper presents an analysis of how to preserve the stability of a manipulator during contact tasks. This work only assumes that the object that will be made contact with is stable in isolation.

Equations are derived for the stiffness of soft finger contacts. The stiffness is found to have an approximately linear relationship with the o vertical depression of the soft finger. The contact stiffness becomes larger (stiffer) with larger depressions. The derivation is based on the power law for soft fingers which relates the radius of circular contact area and the normal force.

An algorithm is presented for calculating the stiffness matrix for fingers experimentally, based on applying known forces and moments, and measuring the resultant displacements and orientations.

The properties of the grasp stiffness matrix are examined. It is shown that a stiffness matrix is conservative if the matrix is symmetric and satisfies a certain differential condition. In general a conservative stiffness matrix is Cartesian space will be nonconservative when transformed into joint space using a configuration dependent Jacobian (and vice versa).

In a study of adaptation to varying force fields during reaching movements, it was found that subjects were unable to adapt to a time-varying force field while they were able to adapt to a velocity-varying field. They speculate that the system that adapts movements to external forces cannot use a temporal representation.

In order to determine the optimal grasp, a series of performance indices were defined. These indices are a stability grasp index (how close the grasp points are to a regular polygon), an uncertainty grasp index (how far away the grasp points are from edges), a maximum force transmission ratio index (based on the force ellipsoid and the desired force direction), a task isotropy index (distance from singularities) and a stiffness mapping-based grasp isotropy index (based on the grasp stiffness). These measures are normalized (by dividing them by the difference between the maximum and minimum possible values) and thus also non-dimensional. Different weights can be given to the different indices depending on the task.

A method is described for planning the necessary stiffness for various grasping and manipulation tasks. The stiffness of the grasped object is related to the stiffness of the joints through the grasp matrix. The desired stiffness geometry for the task in object coordinates can then be transformed to determine the necessary joint stiffness and/or geometry of the hand. Various examples are given.

The adaption to a novel, velocity dependent force perturbation was found to be different depending on the specified goal. When subjects were asked to perform a spatial goal (continue to the target), their movements became curved but returned to reach the final point. In constrast, when subjects were asked to maintain the same effort, the deviation increased throughout the movement, resulting in large endpoint deviations. A significant after effect was only seen with the spatial goal.

During a multiple finger force production task ,the effects of adding and removing fingers was investigated. Enslaving and the magnitude of force deficit were observed. The enslaving was found to be history dependent, while that the master fingers was not. They suggest that this is evidence that deficit and enslaving are produced by different mechanisms.

A frame invariant measure is defined for compliance grasps, and an interpretation of the stiffness matrix is given.

A method is presenting for calculating the stiffness matrix using the Hertz model. They contrast this to the linear spring compliance model that is commonly used but is not supported by experiments, and the coefficients must be determined experimentally.

A method is presented for planning control for compliant motion. Compliant motion is when the manipulator position is constrained by the task geometry. The task configuration is represented by natural constraints, which relate the components of ideal effector force and velocity. A particular ideal control strategy is known as the artificial constraints, these constraints should reproduce the goal trajectory while preserving the natural constraints. In general, the artificial constraints are selected to be orthogonal to the natural constraints. Finally a control strategy is needed to transform the artificial constraints to a real world control strategy.

Experiments on the hand in a plane were used to investigate the spring-like behaviour of the arm. It was found that the neuromuscular system is predominantly spring-like, and they showed how the stiffness can be represented as stiffness ellipses or as stiffness matrices. The stiffness was found to be position and configuration dependent.

The use of the Jacobian pseudoinverse in kinematically redundant solutions can lead to unpredictable configurations, such that the posture at a given position can depend on the path taken to get there. This is due to the nonintegrability of a different equation associated with the inverse. In this paper, they present a class of generalized inverses that are integrable within simply connected regions without singularities. This inverse captures the effect of the nonlinear manipulator geometries on the Jacobian.

A mathematical analysis of force production in pushing a pivoting stick was performed to determine what is required to maintain static stability. The hand rotational and translation stiffness can be used to stabilize the stick. It is suggested that such a strategy is generally used by humans for force-production task. Such analysis can also be useful in tool design.

The act of pushing a wall was modeled for different strategies. They present a static model, as well as a dynamic model based on the assumption that the pushing force is a consequence of nonzero mechanical impedance of the upper limb. It was concluded the by placing the feet apart, there is more control over the centre of pressure (than when feet are placed together), and so this is the best way to control force at the hand.

It is shown that any symmetric positive semi-definite 6x6 spatial stiffness matrix can be written in Lon\vcaric's normal form. The normal form decouples the linear and rotational components as much as possible, and the 3x3 off-diagonal blocks are diagonal. The stiffness properties of different manipulators can be more easily compared when they are all in normal form

The contribution of individual finger forces to moment production on a mechanically constrained handle was studied under varying task parameters. The forces and moments were observed to be different to when manipulation unconstrained objects. Additionally, while the mechanical advantage hypothesis (that the finger force production contribution is dependent how close the moment arm is to the rotation axis) explained some of the data, it was not able to explain other parts. From this they suggest that this theory is limited in its applicability.

The possible stiffness that can be imparted on a 3 DOF variable geometry parallel planar robot is considered. The aim was to obtain a specific stiffness in a given position/orientation of the platform by exploiting the excess degrees of freedom. Arbitrary values of the Cartesian stiffness are unattainable using this method although some of the stiffness elements can be generated.

The use of the derivatives of the Jacobian of parallel robots for stiffness modulation was presented. The derivatives were with respect to the position and orientation. The derivatives were associated with the desired stiffness. The derivatives also allow a geometric interpretation of the singularities.

A model of the stiffness and impedance properties of the arm is presented, based on experimental findings. Predictions are presented for 2D movements in a horizontal plane. The model predicts static stiffness by assuming that the joint stiffness is linearly related to the applied joint torque. From this, the Cartesian stiffness is calculated, and plotted as stiffness ellipses. The dynamic stiffness during movements was calculated after assuming a minimum jerk trajectory and calculating the necessary torques to produce such a movement. The model predicted well impedance geometry from previous studies.

The impedance of the arm was studied while subjects used a virtual tennis system. The ball was virtual, displayed on a monitor, and the subject moved a handle attached to a robot that is impedance controlled. Hand impedance was measured by applying a disturbance at two different timings (during the preparation of the movement). It was found that subject prepare for the hit by increasing hand stiffness, that the stiffness is altered depending on the mass of the ball, and that some subjects were able to alter their arm viscosity in less viscous environments to maintain stability.

A method is presented for measuring the impedance (as well as the stiffness) of the arm. The arm, modeled as two joints, is displacement by a small amount, and the force measured. An equation is written relating the hand inertia, viscosity and stiffness matrices with the force and displacement. Using a least squares method, the value of the three matrices are found at various postures. The hand inertia agreed well with predicted values, the spatial variations in the stiffness matrices were similar to those seen in the previous study of Mussa-Ivaldi et al. (1985), and the stiffness and viscosity ellipses tended to have similar orientation.

The use of artificial force-field based methods for robotics is reviewed. A closed-loop control system, known as a time-based generator, is described for generating a trajectory based on the error vector. An example of control of a unicycle is given.

Some telemanipulation tasks were performed with and without force feedback (with the CyberGlove and CyberGrasp). Although the performance time was not significantly better with force feedback, there appeared to be some other benefits (faster learning, stability).

The usefulness of force feedback during telemanipulation was tested in this paper. Three tasks where performed. During object size discrimination, the subjects performed quite well, approaching the limits of human proprioception. During force regulation, the subjects largely succeeded in controlling the force applied to an object. The test of discriminating stiffness was considered the most difficult.

This paper studied finger force vectors during torque production tasks. The digit forces changed both in magnitude and direction at different torques. At large supination torques, the index and middle fingers generated torques in an opposite direction to the ring and little fingers.

Cyberglove manual

Cybergrasp manual

The difference in grasping an object with different visual feedback was studied. The subjects were shown, before the movement, either the real object, a virtual computer rendered object or a symbolic presentation (using a mirror setup). The visual information was removed at the initiation of the movement. Haptic feedback was provided (using a real object). Different kinematic properties were compared (e.g. preshape aperture, grasp onset latency, movement velocity), and no significant difference was seen between grasping real and virtual objects (as opposed to pantomiming behaviour found in other studies).

A route planning system is described that uses online control to determine the current state without an explicit world model or path plan. The route is planned using a dynamic model in terms of the angular acceleration. The goal acts like an attractor, and the obstacles like a repeller. Multiple objects can be simulated by linear combination. The routes predicted by the model were similar to those performed by humans in a Virtual Reality experiment.

The use of a Cyberglove and a force feedback glove to perform stroke rehabilitation exercises is presented. Its advantages over traditional exercises is that it can be performed alone and data about progress is readily available.

A review is made of haptic feedback devices, including desktop devices, haptic feedback glove, arm exoskeleton devices, workstations and whole body devices.

The difference in grasping a real and a virtual ellipsoid was studied. Grasping the virtual object showed greater deceleration and variability - this is probably due to the lack of contact at the end of the the motion. Furthermore, the type of grasp selected was dependent on the amount of visual information given.

The construction of a data glove with a force sensor is explained. Flex sensors are used to measure the bending of the fingers (using resistors that have resistance relative to the bending angle). The force sensor was made using a steel blade with two strain gauges at each side. The change in resistance is related to the applied force.

A short review of the applications of immersive VR in behavioral neuroscience and other areas is presented. The VENlab at Brown is described along with some applications.

Cyberglove manual

A system was developed for using a CyberGlove along with force sensors to describe the posture and force to apply for telerobotics. It was found that the grip size was primarily controlled by changes in the MCP angle, while the main force exertion is from the thumb and index fingers. It was suggested that these primary parameters can be used to specify the robot grasp.

A technique is presented for modeling the workspace of different limbs based on the joint limits.

A review is presented of the kinematics, the neurophysiology and neuroimaging of grasping in humans and monkeys. The role of the brain areas that may be involved in grasping are considered.

A feed-forward controller is described for redundant systems (i.e., those with excess degrees of freedom). The controller requires knowledge of the Jacobian of the system, which it is assumed can be learned. The Jacobian is of the mapping from elemental variables (e.g., joint angles) to task variables. By altering the relative variances of the elementat variables and the task variables, the model produces predictions for a finger force production task that are consistent with experimental observations and the UCM hypothesis.

Manipulation is described in terms of the combination of closed loop controllers. These controllers are potential functions that relate sensors and effectors. The force and moment errors are used by descending the wrench error gradient. Wrench closure and contact constraints are enforced by projecting these constraints into the nullspace of the control objective. In experiments, the controllers are learnt by reinforcement learning.

This study looked at neurons in F5 in the monkey. They found neurons that have a preference for a grasping an object or set of objects with a particular type of grip. They note that this activity is not related to the individual finger movements. About half of the neurons also responded to the presentation of objects when grasping was not required. It was claimed that the results of this study provide neurophysiological grounding to psychophysical findings that simplifying strategies are used by the CNS in planning grasps.

A description is given of grasping and manipulation in terms of "primitive" regions of the hand (i.e. which part of the finger). These descriptions can be combined. For manipulation, the descriptions consist of the transitions between primitive regions.

A review of hand use in grasping is presented. They consider finger movements from a development perspective and the coordination needed for grasping (e.g., kinematic coordination patterns, force coordination). They also review findings from neural studies in monkeys and humans relating potential control sites and activations and observed movements.