Determine the location an object by image processing used for controlling autonomous vehicles
Trong bài viết này, chúng tôi đưa ra một phương pháp mới xác định vị trí của các đối tượng bằng
xử lý ảnh thu được từ một camera CNN kiểu Bi-i V2. Các đối tượng nằm trên mặt phẳng và có vị
trí bất kỳ đến camera. Đây là ph ương pháp xác định vị trí đối tượng dựa trên cơ sở các quan hệ
hình học trong thực tế và ảnh thu được. Thuật toán lai xử lý trên thiết bị Bi-i bao gồm các bước xử
lý song song dùng CNN và các bước xử lý nối tiếp trên DSP. Thông tin vị trí đ ược gửi tới máy
tính trên xe tự hành để điều khiển xe tiếp cận và nâng đối tượng di chuyển tới một vị trí yêu cầu
mới. Thực nghiệm đã được thực hiện thành công trên robot thí nghiệm kiểu Pioneer 3DX.
5 trang |
Chia sẻ: hoant3298 | Lượt xem: 664 | Lượt tải: 0
Bạn đang xem nội dung tài liệu Determine the location an object by image processing used for controlling autonomous vehicles, để tải tài liệu về máy bạn click vào nút DOWNLOAD ở trên
Phạm Đức Long Tạp chí KHOA HỌC & CÔNG NGHỆ 116 (02): 35 - 40
35
DETERMINE THE LOCATION AN OBJECT BY IMAGE PROCESSING
USED FOR CONTROLLING AUTONOMOUS VEHICLES
Pham Duc Long*
College of Information and Communication Technology - TNU
SUMMARY
In this paper we propose a new method to locating of objects by a camera CNN Bi-i V2. These
objects are on the ground and they have any position with camera. It is method to locate objects
based on geometric relationships of real size of object with its image in camera. The hybrid
algorithms are performed in Bi-i that includes parallel and serial tasks of processing. The values of
distance and orient are sent to an embedded computer on the robot for controlling. Experiments are
successfully implemented on the robot Pioneer 3DX.
Keywords: Estimating location by image processing; camera pinhole
INTRODUCTION
*
Today, the services of goods transportation
(LOGISTICS) is growing due to the
circulation of goods across the globe are on
the rise. To response this development the
topic of automation for warehouse, using
automatic car for transport, ... is interest in
researchers. Have appeared the AGV -
Automated Guided Vehicle. The chain of
freight car is researched. They can
automatically transport, load and unload
goods. These vehicles are usually automated
using ultrasonic sensors or camera image and
encoder to estimate location in the warehouse.
When using a camera and image processing
program researchers have used the method
with one or more cameras [1, 2, 3, 4].
Estimating location of a object by using two
camera is the results better but longer
computation time. In the technique to
successfully a same task if that plan is
simpler, using less equipment than going for
many more outstanding features. The mission
that research done in this paper is to use a Bi-i
V2 CNN camera fixed on a Pioneer 3DX
robot captured the image and processing to
compute for estimating location parameters
(distance and angle orient) of a model of
pallet and send these parameters to industrial
PC for controlling the robot moves it to the
pallet, lifting and moving pallet to the
required position - "region target". In the Fig.
*
Tel: 0912 551589, Email: pdlong@ictu.edu.vn
1 the robot lies on the root of system co-
ordinate that is the point O. The Bi-i camera
fixed on the robot can observation an region
in which the captured image is used for
computed. The position of the object (pallet
model) was placed in the observation space.
This position can be changed from a host
computer (e.g. from a notebook) via
wireless network.
Fig. 1 Robot approaching pallet and take it to area
target
The estimation of location of pallet is
implemented in two cases:
When pallet lies in an area, its real distance
bounded is known.
When pallet lies in the any space.
ESTIMATING LOCATION OF OBJECT
BY IMAGE PROCESSING
From a stationary camera to estimate location
for any object in viewable space of the camera
We use a camera CNN Bi-i V2. It is placed
on a robot. Object's image is captured by
camera in the Fig.2. When robot approaching
to object on direction P - P it uses the point K
as target (Fig. 3). Program of image
processing have to computing parameters and
sends it to Industrial PC:
Phạm Đức Long Tạp chí KHOA HỌC & CÔNG NGHỆ 116 (02): 35 - 40
36
The ordinates of point K (XK and YK), and
The angle orient (angle of line P-P and axis X)
Fig. 2 Object and its image
The point M is a black pixel its ordinate Y is
biggest in the image. In the Fig. 3:
TT is line through optical axis of lens.
NN is line perpendicular with TT.
The camera leans to axis X angle pan.
If we compute to get real distance O1M from
pinhole point O1 to the point M (next step get
the XK and YK), and we already have real size
of the object then we can to compute to get
YK and XK.
The point M is not always on the optical axis
of lens that is often lop-sided (to right or left
of the optical axis - on the Fig. 3 is being
skewed to the right at an angle R).
The perform as follows:
Determining location of the point M:
O1M=O1H*tang( ). (1)
= { R, L}
If the point M is on the right of symmetrical
axis of image (that is XM > 64 pixels):
In real plane then
XM = O1M*cos(pan - R). (2a)
and YM = O1M*sin(pan - R).
If the point M is on the left of symmetrical
axis of image (that is XM < 64 pixels):
In real plane then:
XM = O1M*cos(pan+ L). (2b)
and YM = O1M*sin(pan+ L).
If the point M is on the symmetrical axis of
image (that is XM = 64 pixels) :
In real plane then
XM = O1M*cos(pan). (2c)
and YM = O1M*sin(pan).
After computed the angle orient
XK = XM+(half size of width of the
box)*cos(orient); (3)
YK = YM+(half size of width of the
box)*sin(orient);
Because the root of system co-ordinate O
does not coincide with center of lens O1
therefore
XK=XK+70; (4)
YK=YK+200;
Fig. 3 Model of estimating location from
robot to pallet
Fig. 4 Real side of object is ON’ and its image
(distance ON in the line, which is perpendicular to
axis optical through points CP’,
is angel view of lens)
Fig. 5 Relation of the real side M'N' and its image
MN in the case plane of image is un-perpendicular
with axis optical
Phạm Đức Long Tạp chí KHOA HỌC & CÔNG NGHỆ 116 (02): 35 - 40
37
In the model of Fig.5 we knew in real size of:
- height from ground to camera, declination
of camera tilt, the bounds viewed of camera
(CO, CQ' are knew). Like this the angle of
OCQ' is knew
- Ordinate in direction Y of point Q (YQ).
- Dimension ON keeps takes 128 pixels
(because resolution of camera equal
128x128).
- Dimension OM keeps (128-YQ) pixel.
In image we have the point M far the point O
a distance i pixel. The distance in image is
OM and corresponding real distance is O1M'.
Angle of OCM (with top is point C) has
degree:
i*(angle OCQ'/number of pixel in the distance
OQ) (5)
Real distance from point O1 to point M is
O1M'=height*tang(((tilt -
2
) + i)*(góc
OCQ'/number of pixel in the distance OQ))
(6)
Corresponding to Fig.1 we computing the
distance O1M. Using formula (4) we get XK
and YK
Fig. 6 The variant of real dimension in horizontal
direction.
We need computation HM in Fig. 6. We have:
= atan(((Maxsize-Minsize)/2)/Length)
From Minsize to Maxsize in the view region
size in horizotal direction is increasing a
number Maxzide-Minsize. After each
horizontal pixel size will increase:
(Maxsize-Minsize)/ Length
For this reason, coefficient to correct for get
real dimension in horizontal direction as:
heso_thuc = (Minsize+(128-
MAXy)*((Maxsize-Minsize)/ Length))/128
Fig. 7 Model for computing angel orient.
Let examining case of when object is (and
also other cases when object is not) on the
optical axis of camera:
L and R is denoted the real sloping to the
left and right angle side of object,
respectively. In the image these angles are 'L
and 'R. If optical axis of camera is
perpendicular to object's plane then L = 'L
and R = 'R. In fact, optical axis of camera
often is not perpendicular to object's plane.
Therefore, when by image processing we
compute to get angles 'L and 'R , but their
value is not equal real values of angles. For
get real value of the L and R we have
multiply 'L and 'R with a correct coefficient
k. The value of k is dependent on the
parameters height and tilt (in Fig. 5). Formula
for computing it as follows:
Because L + R = 90
0
then in the image if
( ’L + ’R) < 90
0
then k = 90
0
/( ’L + ’R). This
reason is logical because relations of L on
'L, and R on 'R are linear (however we are
not expressing this relation).
orient = pan- L or (7)
orient = pan+ R - 90
0
Fig. 8 The variant of angles L and R when nearest
point of the object is not on the vertical symmetrical
axis of image
Phạm Đức Long Tạp chí KHOA HỌC & CÔNG NGHỆ 116 (02): 35 - 40
38
Estimating location of an object in any
space (when camera is moving)
In this case the real view region of camera is
unknown. We knew:
+ Real size of the object (by millimeter), and
+ Parameters of array optical sensor in our
camera (width, height by millimeter and
degree resolution of camera), and
+ Len's length of focus. Using model camera
pinhole we can calculate distance from
camera to object in two cases:
When plane of object is parallel to plane that
containing sensors of camera, and optical
axes of camera is identical with the normal of
object's plane in the geometrical center. This
is standard model of camera pinhole. This
case often is in some cases using image
processing receiving control information in
the industrial robot arm.
The case of when object's plane is not parallel
to plane that containing sensors of camera,
and optical axes of camera is sloping to
object's plane an any angle. In practice, this
case is when a camera is placed on an
autonomous robot.
The resolution of the cases i) and ii) is given
in the paper [5].
EXPERIMENTS
The real parameters of a camera (for instance:
length of focus, distort image in the corner of
image,...) often are not accurate with
parameters of the manufacturer; To adjusting
these parameters one realize calibrating for
camera by software Calibration of
MATLAB.
The algorithm
Algorithm of the case 2.1 is shown in Fig. 9
Fig. 9 Algorithm in the case of 2.1
Results
Experiments are realized with two models of
the object:
The box size of 160x160x130 and model of
Korea pallet in the ratio of 1:4
Fig. 10 The model of objects in the experiment
Fig. 11 Integrating an Autonomous Vehicle: :
Router TP LINK W8951ND; : Camera Bi-i :
IPC : Robot Pioneer 3DX
Fig. 12 The robot approaching an object
Phạm Đức Long Tạp chí KHOA HỌC & CÔNG NGHỆ 116 (02): 35 - 40
39
The lens type of 1:1.4 25 mm is used in the
experiments. The speed captured and
processing image is 1000fps. In the
experiments program image processing
exactly computed the location of model pallet
and then sending these information via
wireless communication to an IPC that is
placed on the robot to driving accurately the
robot to model of pallet, lifting and carrying it
to the required position. It has confirmed the
correctness of the proposed method.
CONCLUSION
Using a CNN camera type Bi-i V2 placed on
a mobile robot with wheels can captured
images of an object in the search space to
locating position of the object and provides
these information for the IPC in the robot to
driving it move to object to perform the next
task. When using the camera CNN a lot of
image processing tasks have been parallel
implemented that causes to reducing time for
computing in every task and also reducing
time for computing overall mission. Now, the
trend of computer vision is using stereo-
camera for locating the object. The continue
study of image processing algorithms to
locating the robot will be used 2 CNN camera.
With this approach we can be obtained more
significant results in the near future.
REFERENCES
1. Giovanni Garibotto, Stefano Masciangelo, Marco
Ilic, Paolo Bassino, ROBOLIFT: a Vision Guided
Autonomous Fork-Lift for Pallet Handling, IEEE, pp.
656-663, 1996.
2. Michael Seelinger, John-David Yoder, Automatic
Pallet Engagment by a Vision Guided Forklift,
Internatinal Conference on Robotics and Automation
Baccelona, Spain, pp 4068-4073, April 2005.
3. Daniel Lecking, Oliver Wulf, Bernardo Wagner,
Variable Pallet Pick-Up for Automatic Guided
Vehicles in Industrial Environments, IEEE, pp.
1169-1174, 2006.
4. Sungmin Byun and Minhwan Kim, Real-Time
Positioning and Orienting of Pallets Based on
Monocular Vision, 20th IEEE International
Conference on Tools with Artificial Intelligence,
IEEE, pg 505-508, 2008.
5. Phạm Duc Long, Pham Thuong Cat, Using
Image processing for computation distance and
orient from pallet to autonomous forklif,
Proceedings of the VCCA 2011, Hanoi 12-2011.
TÓM TẮT
XÁC ĐỊNH VỊ TRÍ ĐỐI TƢỢNG BẰNG XỬ LÝ ẢNH
ỨNG DỤNG TRONG ĐIỀU KHIỂN XE TỰ HÀNH
Phạm Đức Long*
Trường Đại học Công nghệ Thông tin và Truyền thông - ĐH Thái Nguyên
Trong bài viết này, chúng tôi đƣa ra một phƣơng pháp mới xác định vị trí của các đối tƣợng bằng
xử lý ảnh thu đƣợc từ một camera CNN kiểu Bi-i V2. Các đối tƣợng nằm trên mặt phẳng và có vị
trí bất kỳ đến camera. Đây là phƣơng pháp xác định vị trí đối tƣợng dựa trên cơ sở các quan hệ
hình học trong thực tế và ảnh thu đƣợc. Thuật toán lai xử lý trên thiết bị Bi-i bao gồm các bƣớc xử
lý song song dùng CNN và các bƣớc xử lý nối tiếp trên DSP. Thông tin vị trí đƣợc gửi tới máy
tính trên xe tự hành để điều khiển xe tiếp cận và nâng đối tƣợng di chuyển tới một vị trí yêu cầu
mới. Thực nghiệm đã đƣợc thực hiện thành công trên robot thí nghiệm kiểu Pioneer 3DX.
Từ khóa: Xác định vị trí đối tượng bằng xử lý ảnh; Camera pinhole
Ngày nhận bài:25/01/2014; Ngày phản biện:10/02/2014; Ngày duyệt đăng: 26/02/2014
Phản biện khoa học:TS. Nguyễn Văn Huân – Trường ĐH Công nghệ Thông tin & Truyền thông - ĐHTN
*
Tel: 0912 551589, Email: pdlong@ictu.edu.vn
Các file đính kèm theo tài liệu này:
- brief_42063_45910_5620141535307_9986_2048634.pdf