By Michael Piovoso, Ph.D., M J Consulting and David Booth, InvenGen Engineering, LLC.

As global competition raises the performance bar in bringing creative products to the markets, companies continue to focus on finding new ways of facilitating fresh innovative streams of thinking to meet these challenges. Manufacturers are now finding that using applied math and statistical models can help them see subtle patterns and structures in their product and process data which provide additional market advantage. Everyday, companies are discovering greater application of these tools on the factory floor. Methods such as:

• wavelet analysis
• partial least squares (PLS) - projections to latent structures
• least squares - discriminate analysis (PLSDA)
• multi-resolution analysis (MRA)
• neural networks (NN)
• principal component analysis (PCA)
• multivariate image analysis
• decision tree analysis and
• design-of-experiments (DOE)

are now being used effectively by manufacturing & quality personnel, engineers and product designers to solve production application issues on the factory floor and create innovation opportunities. Access to seemingly unlimited amounts of data from today's processes presents an occasion for better understanding how they work and how to improve them. Armed with this new insight, improvements are being quickly translated into healthier return-on-investments. Methods once primarily reserved for use by researchers are becoming more and more a new source of product and process innovation on the front lines of manufacturing. This list is not exhaustive and is meant only to provide an overview of the more popular techniques.

Method Explanations and Production Examples

Through skillful application of these methods, a broad range of manufacturing challenges have been addressed such as product inspection and classification, process development and optimization, development of test methods, product verification and validation, product damage detection, and solving day-to-day shop-floor process challenges. First, a brief description of these methods will aid in better understanding their usefulness.

Wavelet analysis, as it applies to multivariate applications, is using wavelet transforms to convert a signal into information about the frequency content at various points in time. That information can be manipulated by projection regression methods for the purpose of identifying underlying structures in the data or alternatively, can be a preprocessing step in fault detection, product quality or other pattern recognition problems.

Partial least squares (PLS) is a projection regression method used to find relationships between process variables and their responses.

Applied Analysis Tools

Partial least squares - discriminate analysis (PLS-DA) is a similar projection regression technique that is used to discriminate between classes of process variables such as different treatments of similar test articles.

Neural Networks (NN) is a nonlinear regression method that is inspired by neural biology and is useful for finding nonlinear relationships between process variables and their responses. The first four analytics are discussed in greater detail in the next section.

Multi-resolution analysis (MRA) is a multivariate approach to object detection especially in the context of large data bases or object matching/ detection in images of different scales. This approach first permits decomposition of information into different scales, and second, allows the use of fast and stable algorithms. When combined with wavelet analysis, MRA has been applied to:

• photo-analysis
• automated tasks such as features extraction and segmentation/classification
• production-line part inspection
• speech analysis & improving for performance of voice recognition algorithms
• video segmentation to improve frame transitions
• image enhancement of arterial and vein networks
• identification of small bowel obstructions from gastrointestinal sounds
• electrocorticogram (ECoG) records from epileptic patients
• classifying ECG arrhythmias
• financial trend detection and
• home-land security

to name just some of the applications. Applying MRA in production may also include:

• part sorting algorithms to aid robotic part transfer
• on-line non-destructive product testing
• smart quality compliant systems
• for security systems that recognize employees, voice and computer keystroke patterns.

New product technologies may include software to help non-video editing savvy users create more imaginative presentation of ideas by making it easier to combine video clips, 3D solids modeling and computer graphics. MRA can be used to enhance display imaging on advertising platforms ranging from iPods to outdoor bill boards or to enable new phone camera features with active image recognition for smart picture editing. Medical imaging technology can be enhanced by MRA in the areas of diagnosing skin and tissue diseases.

Principal Component Analysis (PCA) - simply put, PCA is a way of identifying patterns in data, and expressing the data in such a way as to highlight their similarities and differences. Like the previous mathematical procedure, this linear transformation converts a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables called principal components.

Applied Analysis Tools

The transform does this by choosing a new coordinate system for the data set such that the greatest variance by any projection of the data set comes to lie on the first axis (then called the first principal component), the second greatest variance on the second axis, and so on. The first principal component accounts for as much of the variability in the data as possible, and each succeeding component accounts for the remaining variability. The objective of using this method is to discover or reduce dimensionality of a data set or to discover new meaningful underlying variables. PCA does have some important limitations. One of most important is that the technique is linear; therefore any non-linear correlation between variables will not be captured.

Considered to be the work-horse of multivariate analysis, PCA is a component part in many of the techniques described here. For example, PCA is used together with PLS (projection to latent structures) as a first-pass filter to identify the dominant factors in a PLS analysis. PCA is also used effectively in applications which clarify images for better machine vision and classify types of whiskey along with its contaminants and dilutants. Image clarification with PCA is also the first process step used in multivariate image analysis (MRA). PCA is combined with neural networks to also do feature extraction as in the case of face detection ahead of face recognition.

Multivariate Image Analysis is similar to MRA (multi-resolution analysis) in that it is involved in the analysis of data within complex images. It differs, however, that it deals with multivariate data (data obtained by measuring a number of different quantities simultaneously) present in images. Image data when collected in multiple spectral bands produces a multivariate image. Multivariate images can have spatial, intensity, spectral and time (temporal) resolution. MIA can provide classification capabilities when PCA is used to pre-process the data.

MIA has been found useful in medicine (in-vivo examination of tissue), forest valuation, satellite image inspection and quantitative chemical imaging techniques to understand the structure and composition of formulated products. In production, MIA can be used with manufacturing of optical systems to check for unwanted patterns in illumination, automated on-line parts inspection and checks for surface defects on parts. Care should be taken, however, in insuring computational issues are manageable due to the size of the images.

Multivariate Decision Tree Analysis is a method that aims to find compact rules describing data sets well by devising prediction rules, screening the variables and summarizing the multivariate data set. This method features a tree-like way of representing a collection of hierarchical rules that lead to a class or value. The approach uses principal components of the data set as inputs into the decision tree analysis. PCA pre-processes the data identifying and helping to remove outliers, deals with missing variables, and identifies principal components. The decision tree algorithm is then applied to the principal component scorings using the PCA loadings to interpret the information. Software applications are available that provide strong decision tree algorithms.

This method results in decision trees that can be used for risk assessment, financial analysis, medical diagnosis, system fault checking, classification problems and product & process validation.

Design of Experiments (DOE) is a strategy for setting up blocks of experiments in which all variables are varied in a systematic manner for the purpose of determining the correlation between variables and to predict results. It is the most efficient approach for organizing experimental work. DOE selects a diverse and representative set of experiments in which all factors are independent of each other despite being varied simultaneously. As a result, a model is defined that can show the importance of all factors and their interactions and predict process outcomes.

DOEs are invaluable in all aspects of product and manufacturing process development. Screening DOEs help in finding the experimental factors which dominate the response of the process. Optimization routines help in finding the best process settings or design variable levels that take into account all the demands of the responses. Finally, there are experiments which help find the level of variables to insure process or performance robustness.

A specific production example highlighting the above methods includes the case of a large pharmaceutical and medical device company needing to insure a high standard of product quality for surgical knives. To provide this level of product performance, manufacturing operators destructively tested knife edges throughout the sharpening process to assure optimum process settings and performed a 100% inspection of the blades to prevent non-conforming product from reaching the customer.

Defective product levels were reduced and the knife sharpening process was improved by conducting simple screening design-of-experiments to isolate the factors that dominated the sharpening process. Modeling experiments were then used to optimize the process by finding robust settings for the factors that provided greater manufacturing control. An added result was a sharpening process that required less intervention by the operators thereby reducing further levels of impressed error and noise. Combining Wavelet analysis and partial least squares - discriminate analysis provided a unique opportunity to classify blade performance non-destructively. A novel method of combining physical features and attributes of the blade was used to create a ‘good' blade model against which other blades were classified correctly 97% of the time. PLSDA was also used to classify the blades for surface defects.

Expanded Descriptions of a Few Important Methods

By leveraging ubiquitous factory-floor computing power, sophisticated multivariate analysis software applications are assisting manufacturing and engineering professionals in analyzing production data to find process trends and latent data structures that hold the key to valuable product and process improvements.

Wavelet Analysis - Similar to Fourier transforms, wavelet transforms are very effective as a spectral filter to model manufacturing data and have the trademarks of good compression and de-noising of complicated signals. Data models, in the form of matrices of wavelet coefficients can be employed along with PLSDA to classify model membership of production pieces.

Wavelet analysis is a technique used to transform an array of N numbers from their actual numerical values to an array of N wavelet coefficients. In one perspective, wavelet analysis can be seen as a ‘refinement' of Fourier analysis.

The basic Fourier transform highlights the spectrum of a function, image, signal, etc., but globally vs. locally. The wavelet transform provides localized frequency decompositions which yield information not only on what frequency components are present but also when or where they are occurring. The "localized" nature of the wavelet transform allows you to easily pick out features in your data such as spikes (for example, noise or discontinuities), discrete objects (in, for example, astronomical images or satellite photos), edges of objects, etc. The wavelet transform can be thought of as a band-pass filter, where the location and width in Fourier space depends on the wavelet scale. Larger scales imply a lower frequency and small bandwidth. Unlike windowed or short-term Fourier transforms, automatic and appropriately adjusted localization adds to the flexibility of wavelet transforms and highlights their popularity.

Wavelet analysis has been expertly employed in production to determine if the combustion in a process flame is stable or unstable. In another application, two unlikely candidate variables, near-infrared spectra of cellulose sheets and the viscosity of the cellulose sheet powder were found through wavelet analysis and subsequent PCA treatment of production data to be dominate process factors to give better control of manufacturing resulting in higher process yields. Other examples of novel use include automated medical needle inspection and classification which reduces the cost of labor for these operations, classifying the performance of automobile sun roofs by the sound of operation, and surface & edge defects in metal surgical instruments. Wavelet transforms have also been combined with neural networks to provide even greater classification capability in applications such as laser welding [Luo, 2005]. In this application, the sound of the welding is processed with wavelet transforms and then analyzed by the neural network against a pre-trained model. Through these and similar applications, companies have been able to create hundreds of thousands of dollars in value by employing wavelet analysis in the creation of trade secret technologies and production test methods.

Partial least squares - projections to latent structures (PLS)- Data, production or otherwise, contains valuable information, some of which is easily identified. Usually, in science and industry applications, regression models are one of the most common ways of relating one or several dependent variables, responses (Y), to independent variables, predictors (X). Multivariate linear regression typically has been used to model Y by means of X which works well as long as X variables are few and fairly uncorrelated. With an explosion of data from modern instrumentation, (spectrometers, bio-analytical platforms, sensor batteries, etc.), now X variables tend to be many, noisy, correlated and incomplete.

PLS allows investigations of problems with greater complexity and analyzes data in more realistic terms. This multivariate analysis technique features separation of the ‘signal' in the data of a complex system from the ‘noise' and allows the results to be presented in graphics that are easily interpreted.

PLS facilitates analysis and visualization of large complex datasets and provides a holistic summary of the data:

• Representing correlations and trends pictorially
• Separating systematic behavior from noise
• Handling missing data
• Detecting outliers
• Highlighting clusters and patterns

Regression by means of projection to latent structures (PLS) is used widely as a chemometric data analysis tool in the chemical industry but is recently finding application in generic manufacturing operations as engineers become skilled with its use. The term ‘latent structures' refers to relationships between predictors (X) and responses (Y) many times too subtle and global to be understood by merely observing the data or through the pedestrian use of statistics. An example of this method's use in industry includes relating Y - quality and quantity of manufactured products to X - the conditions of the manufacturing process.

Some examples of where PLS has been successfully used are with silicon wafer processing to distinguish between future good and bad batches. Nineteen variables are measured on-line from a wafer etching tool in five process steps to predict the performance of the chips. In this and other similar cases, dynamic PLS can reduce dozens of trended process variables down to two or three to simplify control charting. There are also special treatments with PLS multivariate methods to handle non-linear process challenges.

Partial least squares - Discriminate Analysis (PLSDA) -

PLSDA is another projection regression method where X predictors of different classes are modeled together with a dummy Y response variable. Different classes can consist of product that is minimally acceptable, mid-range acceptable, and on the high-side of acceptable. Classes can also be digital in the sense that product is either good or bad, or they can represent categories that may overlap such as oranges, tangerines and tangelos. The results are correlations that create a model against which product can be classified as acceptable or rejectable in production or sorted into like groups.

PLSDA has been used effectively in combining qualitative and quantitative data streams. Seemingly unrelated streams of data have been merged together using this type of analysis to yield greater correlation and classification power. New techniques are being employed to combine a surgeon's perception of the sharpness of a knife along with quantitative blade sharpness data to produce a more robust model of the surgeon/ knife interaction. Another example is combining data from taste tests of potato chips with process test data of the snack food to create a model that can predict how that group of tasters will perceive the product.

Neural Networks -

Neural Networks are a computational paradigm that is inspired by neural biology but not a model of the brain. Just as a neuron, receives information from other neurons, process that data and passes on information to other neurons, so do the nodes in a neural network. The receive information either from an input data stream or from other neurons. These data are process by taking a linear combination and nonlinear transforming that result to form the node output. Unlike PLS in which the model is a one-step process, neural networks are trained recursively. The weights for forming the linear combination are initially chosen as small random numbers. They are then modified in such a way that ability to produce a desired response from a training set is continuously improved. After sufficient iterations, the network converges to a solution.

Neural networks offer an alternative to PLS and PLSDA for modeling. The are particularly powerful in pattern recognition problems and when coupled with process control. Neural networks have been used in such diverse applications as detecting explosive materials to automated driving of a vehicle. They have been used to control a thermoplastic tow placement process for manufacturing composite materials (Heider et al). Other applications include use as a "soft sensor" to estimate difficult-to-measure product properties.

Summary

Product and manufacturing process innovation can benefit from proper application of appropriate mathematical and statistical models. Understanding gained from latent structures in data is now being leveraged in manufacturing settings where it once was mainly used in research. Across the world, companies are stepping up to the challenge of effectively using these important tools to create new opportunities to improve their businesses and stay competitive in global economy.

References

1. Luo,H., et.al., (2005). ‘Application of artificial neural network in laser welding defect diagnosis,' Journal of Materials Processing Technology, 170: 403-411
2. Heider, D., Piovoso, M. J., and Gillispie, J. W., "Application of Neural Network to Improve an Automated Thermoplastic Tow Placement Process," Journal of Process Control, vol. 12, 2002, pp. 101-111.

About the Authors

Mike Piovoso is a principal of MJ Consulting (an affiliate of InvenGen Engineering) and an assistant professor of electrical engineering at Penn State University graduate school in Malvern, PA. and the University of Delaware. He also is a senior research associate with E.I. DuPont De Nemours & Co. He holds a doctorate of Electrical Engineering from University of Delaware and is a registered professional engineer in the state of Delaware.

He has consulted in control system and digital signal processing, AI based systems, spectral estimation techniques, digital control systems, classical analog and digital control theory, and process manufacturing and fault detection for Fortune 500 companies.

Mike has received numerous awards - Penn State Outstanding Teaching Award, Penn State-Great Valley Research Award, DuPont Innovation Award, Two DuPont Silver Medals for technical achievement in various subject areas such as analog and digital control applications excellence, the 1999 IEEE Control Systems Technology Award and the Lifetime Achievement Award from the Institute of Electrical and Electronics Engineers in the area of control systems, and more. He is the author of publications and holds patents in artificial intelligence systems and methods, process monitoring, etc.

Dave Booth is a principal of InvenGen Engineering and experienced both as a technical leader and manager skilled in medical device development, manufacturing implementation, and applied product R&D program management. His career as an engineer and engineering manager has spanned over 25 years in the industry, during which he has created state-of-the-art product designs and developed manufacturing processes for OEM automotive & aircraft parts, commercial electronics, and over 20 medical devices involving multiple million dollar sales volumes annually. As an inventor, his intellectual property includes both disposable and reusable medical devices as well as production equipment patents and manufacturing process trade secrets.

He holds two Master Degrees (business & engineering) from Penn State University and a Bachelor of Mechanical Engineering from Villanova University. He is a registered professional engineer in the Commonwealth of Pennsylvania and a certified manufacturing engineer through the Society of Manufacturing Engineers. Additionally, Dave has published and presented his work in artificial intelligence systems and engineering organization and design management. He is also certificated in Kirton's Adaption/Innovation Theory.