Automated Tissue Scanning

marblefreedomAI and Robotics

Nov 14, 2013 (3 years and 8 months ago)


Automated Tissue Scanning
Brady King
Archie Kinney
Luke Reisner
Dr. Abhilash
PandyaProblem Statement
 Intraoperative tissue classification is slow
and difficult
 Human-performed biopsy (20+ minutes)
 Surgeon makes final decisions
 Tissue classification could allow faster,
more accurate resections
 Better treatment of cancer or other procedures
 Faster recovery time, lower costOur Proposal
 Develop a robotic surgery system capable
of classifying tissue in near real-time
 Use Raman spectroscopy
 Automate the process of
sample collection
 Interface with current
image-guided surgery
 Develop novel interfaces for configuring scans
and presenting results to the surgeon(bloop-bloop!)
The Aesop 3000
 “Automated Endoscopic System for Optimal
 7 degrees of freedom (4 active joints)
 Control schemes: voice, hand,
foot, touch screen, manual, and
 Moves end-effector through a
pivot point
 Only moves up/down/left/right
and in/outObjectives
 Objective 1
 Integrate a rigid robotic arm, the Aesop 3000, into
our current image-guided surgery system
 Objective 2
 Integrate a Raman probe with the robotic system
 Objective 3
 Modify the Aesop 3000 to facilitate automated
tissue scans
 Objective 4
 Develop a novel interface for both the selection of
scan parameters and presentation of scan dataObjective 1
 Integrate a rigid robotic arm, the Aesop
3000, into our current image-guided surgery
 Reverse engineer the Aesop to
find control pins and voltages
 Breakout box
 Develop a system to relay the tracking
information to a PC
 Integrate the tracking information with our
current image-guided surgery systemBreakout Box
 Fearlessly sliced a $2,500 Aesop
control cable in half
 Connected 2 x 55 wires to a PCB
 Made a ribbon cable for routing signals
 Mounted on a sturdy metal platform
 Big thanks to David Sant!Breakout Box PicturePin Determination
Pin Use
 Tested all 55 pins to
1 Ground
determine their
2 Linear up control
functions and working
3 Linear down control
4 Linear potentiometer
 Power, motor control,
5 Shoulder potentiometer
6 Elbow potentiometer
feedback, encoder
7 Ground
feedback, etc.
8 Shoulder/elbow CCW control
9 Shoulder/elbow CW control
10 9.48 V DC
11 …Potentiometer Feedback
 Developed a system to read potentiometer
feedback of 5 most important joints
 Some joints have broken or no pots
 Used a 12-bit USB A/D converter to send
the feedback voltages to any PC
 Can be logged to a file
 Had to buffer linear motor feedback to
prevent automatic shutdownPotentiometer SystemDH Parameters
 Measured Aesop’s link lengths, joint limits,
and other parameters
 Derived the Aesop’s kinematics model
using the standard DH notation
i a d α θ
i i i i
0 0 d 0 0
1 0 0 + (d ) 0 0
2 a 0 0 0 + (θ )
2 2
3 a 0 90° 0 + (θ )
3 3
4 0 0 90° 90° + (θ )
5 0 d 90° 180° + (θ )
5 5
6 a 0 -90° 90° + (θ )
6 6
7 0 -d 180° 0 + (θ )
7 7Kinematics Model
Note: Diagram is
more complicated
than it looks.Tracking in Matlab
 Robotics toolbox for Matlab used to
implement the Aesop’s DH model
 Program reads logged pot voltages
 Function converts voltages to joint
 Modified plot function displays the robot’s
motion in 3D using forward kinematicsTracking Demo!
 Enjoy the demo of Matlab tracking the
Aesop’s movements
(Note from Luke: If it doesn’t work, it’s Brady’s fault.)Objective 1 Challenges
 Aesop 3000 is difficult to work with
 No documentation
 Disassembly, automatic shutdown, etc.
 Some joints have missing or broken
 Will use encoders
 Connectors made by different companies
 Breakout box was more difficult to make
 David Sant helped us with thisObjective 1 Changes
 Haven’t integrated with image-guided
surgery system (yet)
Decided to use encoders
Waiting for motion controller
 Getting motion controller now will save
significant time in Objective 3
Integrating potentiometer feedback would
be a waste of timeIntermission
(I spent way too much
time on this slide)Objective 2
 Integrate a Raman probe with the robotic
 Physically attach the Raman probe
 Integrate Raman classification software with
our image-guided surgery system
 Perform a human factors study
 Further develop classification software
 At this point, the system will be ready to
perform simple point classificationSend Arm Location
Request Arm
Objective 2 System Diagram
Request Raman
Send Raman PointPortable Raman Probe
 Acquired portable Raman probe (finally!)
 Needs to be tested on actual tissue
 Eventually will be mounted on the
AesopNeural Network Classification
 Part of our Raman data classification
algorithm to identify cancer, etc.
 Completed C++ implementation of neural
network forward pass
 Supports a variable
number of neurons and
activation functions
 Will be integrated with the
Raman server applicationRaman Server Application
 Acquires end-effector location from a robot
server (for the MicroScribe)
 Queues and retrieves data points
 Sends data points to 3D Slicer clients
 Need to communicate with Raman probe
 Need to implement Raman data processing
 Pre-processing (noise filtering, background
fluorescence subtraction, normalization)
 Peak extraction, neural network identificationRaman Server DiagramObjective 2 Challenges
 Raman probe still needs more testing
 Compare with previous
Raman data
 Certain algorithms
difficult to implement
outside of Matlab
 Bundled Raman probe software may helpObjective 3
 Modify the Aesop 3000 to facilitate
automated scanning motions
 Physically modify the Aesop 3000 to better
facilitate movement in vivo
New sensors, motors, etc.
May switch to Zeus if insufficient
 Develop software to control scanning motion
Model robot dynamics
Order/build motor controllerMotor Research
 Determined what motors are used in the
Aesop 3000
 Various brush DC motors from MicroMo
 Researched motor specifications
 Voltage and power requirements, etc.
 Determined what encoders are used and
their specifications
 Rotary encoders of various
resolutions from US DigitalMotion Controller
 Ordered motion controller (we think) from
Galil that can handle our motors/encoders
 Standalone Ethernet device
 24 V, 12 A power supply
 4-axis, 200 W amplifier to
drive the motors
 16-bit A/D daughterboard for reading pot
feedback of passive jointsObjective 3 Challenges
 Aesop has only 4 active joints (3 useful)
 Could enhance Aesop or switch to Zeus
Current work will transfer over
 Complete control of the robot arm is difficult
 People at Intuitive Surgical are jerks
 Have to set up a standalone motion controller
 Automatic shutdown may be an issue
 Positioning accuracy hard to predictRevised Budget
# Item Cost
1. Aesop 3000 (or Zeus)
Free ($60k-1M)
2. Breakout box for reverse engineering Aesop
3. System for Aesop tracking
Free ($120)
4. Raman probe
Free ($50k)
5. Raman probe mounting hardware
6. Motion controller (with supply, amplifier, A/D)
7. Robot modifications
8. Two graduate students’ tuition, benefits, and
stipend (3 years)
Total: $2,905Current Progress
Objective 1
Objective 2
Objective 3
Objective 4
Taking Over the
0% 20% 40% 60% 80% 100%Future Timeline
Task Description Date
Track the end-effector of the Aesop
Obj. 1 May ’06
Integrate the Raman probe with the
Obj. 2 March ’07
robotic system, human factors
Modify the Aesop 3000 to facilitate
Obj. 3 November ’07
automated scans
Develop novel interfaces for
Obj. 4 January ’09
automated scans, human factorsFeedback
Any questions, comments, or suggestions?
(Other than the obvious question)