Skip to content
Snippets Groups Projects
expeV2.tex 4.77 KiB
Newer Older
even's avatar
even committed
\section{Experimental validation}

\label{sec:expe}

The main goal of this work is to detect straight segments enriched with a
quality measure through the associated width parameter.
In lack of available reference tool, the evaluation stage first aims
at quantifying the benefits of the new detector compared to the previous
one in unsupervised context on synthetic data considered as a ground truth.
Then comparisons are made with a well established recent detector
\cite{LuAl15} in order to check that global performance (processing time,
ground truth covering, detected lines count and mean length) are not
degraded.

%The process flow of the former method (initial detection followed by two
%refinement steps) is integrated as an option into the code of the new
%detector, so that both methods rely on the same optimized basic routines.
%During all these experiments, only the blurred segment size and its
%orientation compared to the initial stroke are tested at the end of
%the initial detection, and only the segment size is tested at the end
%of the fine tracking stage.
%All other tests, sparsity or fragmentation, are disabled.
%The segment minimal size is set to 5 pixels, except where precised.

The first test compares the performance of both
detectors on a set of 1000 synthesized images containing 10 randomly
placed input segments with random width between 2 and 5 pixels.
The absolute value of the difference of each found segment to its
matched input segment is measured.
On these ground-truth images, the numerical error on the gradient extraction
biases the line width measures. This bias was first estimated using 1000
images containing only one input segment (no possible interaction)
and the found value (1.4 pixel) was taken into account in the test.
\RefTab{tab:synth} shows
slightly better width and angle measurements for the new detector.
The new detector shows more precise, with a smaller amount of false
detections and succeeds in finding most of the input segments.
Other experiments, also available at the {\it GitHub} repository, show
that the new detector is faster and finds more edges than the previous one.


%\begin{figure}[h]
%\center
%  \begin{tabular}{
%      c@{\hspace{0.1cm}}c@{\hspace{0.1cm}}c@{\hspace{0.1cm}}c@{\hspace{0.1cm}}c}
%    \includegraphics[width=0.19\textwidth]{Fig_synth/statsExample.png} &
%    \includegraphics[width=0.19\textwidth]{Fig_synth/statsoldPoints.png} &
%    \includegraphics[width=0.19\textwidth]{Fig_synth/statsoldBounds.png} &
%    \includegraphics[width=0.19\textwidth]{Fig_synth/statsnewPoints.png} &
%    \includegraphics[width=0.19\textwidth]{Fig_synth/statsnewBounds.png}
%    \begin{picture}(1,1)
%      \put(-310,0){a)}
%      \put(-240,0){b)}
%      \put(-170,0){c)}
%      \put(-100,0){d)}
%      \put(-30,0){e)}
%    \end{picture}
%  \end{tabular}
%  \caption{Evaluation on synthesized images:
%           a) one of the test images,
%           b) output blurred segments from the old detector and
%           c) their enclosing digital segments,
%           d) output blurred segments from the new detector and
%           e) their enclosing digital segments.}
%  \label{fig:synth}
%\end{figure}
\begin{table}
\centering
\input{Fig_synth/statsTable}
\caption{Measured performance of both detectors on a set of synthesized images.
$S$ is the set of all the input segments,
$D$ the set of all the detected blurred segments.}
\label{tab:synth}
\end{table}

The next experiments aim at evaluating the performance of the new
detector with respect to CannyLines detector \cite{LuAl15}
on the 102 annotated ground truth images of the York Urban data base
\cite{DenisAl08}.
%One of them is displayed on \RefFig{fig:auto}.
Compared measures $M$ are execution time $T$, covering ratio $C$,
Detected lines amount $N$, total lines length $L$ and ratio $L/N$.
To compare the execution times, for each image of the data base we measure
the execution time of 100 detections, gradient extraction included.
The covering ratio compares the length of ground truth lines covered
by the detected lines wrt the total length of ground truth lines.
A eight neighborhood of 8 pixels is added to lines provided by CannyLines.

\RefTab{tab:canny} gives the achieved results.
\begin{table}
\centering
\input{Expe_auto/cannyTable}
\caption{Measured performance of the proposed detector and CannyLines
on standard images.}
\label{tab:canny}
\end{table}

The new detector shows equivalent performance in terms of ground trouth lines
coverture. CannyLines is faster and finds less but longer lines, but these
results remain quite comparble so that we can arg that the global performance
is not degraded. And most of all, our detector provides an indication on the
detected lines quality through the additional thickness parameter.