Chapter Fifteen
Image Processing and Visualization

This chapter is available as free (personal) off-print from the new book version.

15-01 Introduction

asically all digital images are processed in one kind or another. In fact some old analog images will be processed for presentation today (Figure 15-01). Image-processing was already on its way to becoming an esta­bli­shed field of research in its own right when clinical MR imaging equipment be­came available in the early 1980s.

Figure 15-01:
When reading MR images, you always should be prepared for the unexpected. Digitized imaging brings more of the unexpected into your life. Still there are some easily recognizable features in most images. You recognize them immediately:
(a) When, where, and from where was this picture taken and what does it show?
(b) When and where was this picture taken?
Click to get the answers.

The digital nature of MR imaging, coupled with a wide range of applications, spurred an enormous activity in image-processing of MR imaging data in the last forty years. Computer assisted detection or diagnosis (CAD) systems were de­ve­lo­ped to find and highlight suspicious regions or structures in images of the hu­man body by pattern recognition. They are employed in, e.g., the search for breast, lung and metastatic cancer.

In this chapter we give a brief description of image-processing techniques that have been applied to MR imaging. We also describe the somewhat related field of visualization techniques, with a special focus on 3D visualization methods. Some of the methods mentioned here relate directly to the next chapter on dy­na­mic imaging.

 Involuntarity, image processing can add to the existing delusion and bias in image reading and lead to preconceived but wrong diagnoses (Figure 15-01 and Figure 16-01). Unrecognized optical and mental illusions caused by artifacts created by image-processing algorithms may amplify such errors (Figure 15- 02). These problems are beyond the scope of this introduction to MR imaging; they are treated in detail elsewhere [⇒ Frisby].

Figure 15-02: Top: The coffee house wall illusion – in rea­li­ty all lines are parallel. Bottom: Fraser's spiral – in reality there are only circles.

"Catch as catch can" in daily life might be similar in artificial intelligence: A short excursion into "CAD as CAD can".
A comment.

Gray Scale and Color Images. All digital images are per se gray scale images. They can be "artifically" colored (pseudo-colors). MR images are always gray-scale ima­ges; colored MR images are only used for public relations purposes. Pictures of a number of MRI offsprings (e.g., MR angiography, dynamic contrast-enhanced MRI, functional MRI, MRI tractography, PET-MRI fusion images) often contain over­layed colored areas representing mostly the lower resolution imaging tech­ni­que.

The introduction of color images is a recurrent and lasting topic in diagnostic imaging. However, the contribution of colors to imaging diagnostics, in particular high-resolution images, is much debated because colors do not add any provable diagnostic facts to an MR image. On the contrary, they might confuse, bias, and lead to a loss of information. Colors are subjective qualities. In general, their per­cep­tion is not well understood. In diagnostic imaging, colors lack the dynamic range of gray scale ima­ges, image windowing, for instance, is not possible (cf. Chap­ter 9.).

Another problem is the human eye. In the central part of the retina, there are ap­pro­xi­ma­te­ly six million cone cells which are responsible for color vision. However, 8% of the male population and 0.5% of the female population in Europe and North America suffer from a color vision deficiency. The most common one is deu­ter­ano­maly. People with this kind of relative color blindness perceive green, red and purple as a grayish shade. Radiologists or other physicians among this group reading color-coded images are unable to identify red or green colored areas of these images (a simulation is shown in Figure 15-03).

Figure 15-03: Top: Regular T1-weighted gray-scale MR images of a healthy volunteer. Center and bottom: PET-MRI fusion images of the same per­son. The upper row of the pseudo-color ima­ges shows the normal images; the lower row shows the same images as seen by a person with red-green deficiency (deu­ter­ano­maly).

15-02 Some Fundamentals

The main objective of medical image-processing is to facilitate the gathering of or provide diagnostic information not easily seen, or not seen at all, on un­pro­ces­sed images. In general, the processing of digitally acquired images is aimed at im­prov­ing pictorial information for human interpretation and (or) processing data for autonomous machine perception. Both aims have been targeted in mag­ne­tic re­so­nan­ce imaging and in both areas successful applications have been found.

All computed image-processing requires digitized imaging. In digitized ra­dio­lo­gy, the equivalent of a regular x-ray is taken and digitized directly by a spe­cia­li­zed x-ray system. In nuclear medicine, CT, and MR, imaging slices through the human body are acquired and subdivided into volume elements. The numerical signal from each voxel, in turn, can be translated into a distinct shade of the gray scale and be represented as a picture element in the final image (Figure 15-04). Both single images or a series of similar images can be manipulated, e.g., by noise re­duc­tion, edge or contrast enhancement [⇒ Godtliebsen].

Figure 15-04: Numerical image date output (left) turned into gray-scale image (right). Typically, one finds medical images with an image matrix of 256×256, 512×512, or 1024×1024 and 256 gray levels.

In multichannel imaging, several channels representing n different pa­ra­me­ters can be acquired simultaneously or by consecutive procedures, leading even­tu­al­ly to n images of exactly the same object (Figure 15-05).

Figure 15-05: If multichannel images of the same object are properly aligned, it is easy to compare or compute their signal intensities.

Image-processing allows the connection of picture element data of the same lo­ca­tion in different images with changed parameters; known connections can be com­put­ed, e.g., by using appropriate equations. Such procedures may extract ad­di­tio­nal information or allow quantification of data and thus an objective de­fi­ni­tion of structures, tissues, or metabolic processes. Several single images can, e.g., be add­ed to a new multispectral image (synthetic image) which does not necessarily add useful information or even depict reality (Figure 15-06). Details on image pro­cess­ing can be found in a number of monographs, such as [⇒ Gon­za­lez, ⇒ Russ].

Figure 15-06: Examples of multichannel images: (a) proton-density-weighted, (b) T1-weighted, and (c) T2-weighted images of a slice through the brain. The anatomic location of the pi­xels is ex­act­ly the same; according to ima­ge weight­ing the pixel representation is dif­fe­rent. (d) is a pixel-by-pixel compilation of images a-c. This synthetic image does not reveal any additional diagnostic in­for­ma­tion.

There are different ways of classifying image-processing techniques, for in­stan­ce, they can be defined by what they are supposed to achieve. Types of tech­ni­ques include noise reduction, image segmentation, feature extraction, and clas­si­fi­ca­tion.

Whereas noise reduction is of vital importance for more noisy modalities like ultrasound, MR imaging has, due to a rapid development of MR hardware and software, not the same need for such techniques, perhaps with the exception of dynamic imaging (Chapter 16).

However, image segmentation and image classification have found much more widespread use in MR imaging, partly due to the possibility of acquiring multi­channel data suitable for such processing.

Another important group of image-processing techniques is image registration or image alignment, which sometimes employs image-segmentation techniques to align images. Image registration is important for aligning multimodality data (for instance, nuclear medicine data and MR imaging data from the same pa­tient) or registration of time series.

A specific type of time series (dynamic contrast-enhanced MR imaging) is de­scrib­ed in Chapter 16. Furthermore, time series can also be used for monitoring tumor growth or growth of bones in children.

Another important type of MR imaging time series is BOLD functional MR imag­ing (fMRI) of the brain. Image registration is now routinely performed on fMRI data.

Improvement of diagnostic performance to reduce the ever existing level of uncertainty is one of the main propelling forces in diagnostic imaging research.

Since the human visual-perception system is unable to perform multichannel analysis in order to achieve a new dimension, image-processing developed in pa­ral­lel to the introduction of MR imaging as a clinical tool. Researchers wanted to de­tect any message, possibly hidden, in a single MR image or a series of MR ima­ges. Given that only minimal information existed about how to approach this scien­ti­fic problem, the course of research was mostly empiric.

Historically, the following main lines of approach were followed:

 subtraction or overlay (superposition, data fusion) of multichannel
  images;
 quantification of MR parameters, i.e., T1, T2 and proton density;
 image segmentation and multispectral analysis;
 3D visualization.

Answers to the questions asked in Figure 15-01:

These questions were just asked to confuse you. You might have thought the answers were easy. Of course, they were.

 The picture to the left.
Where: A bird’s-eye view of Central Park in Manhattan. Wrong: This is a vodka commercial with a vodka bottle which looks like Central Park. This picture has been image-processed.
Even if you believe that you know what you are seeing – think twice.

 The picture to the right.
When: Correct – before World War II (in 1928). Where: You are wrong – not in Chicago (even if there is a commercial for the Chicago Daily News), but in Berlin (at the corner of Unter den Linden boulevard and Friedrich Strasse in the city center).
Always check the patient’s history before you read your images and make a diagnosis.

Back to Figure 15-01.