The contribution of Augmented Reality to a zero defect assembly

(1)

The contribution

of

Augmented

Reality to a zero

defect assembly

G. Epe

Research to the integration of AR technology within

assembly processes at SEW Eurodrive

(2)

(3)

The contribution

of Augmented

Reality to a zero

defect assembly

by

G. Epe

Research assignment for the department TEL at the Delft University of Technology

Student number: 4166590

Project duration: September 26, 2017 – February 12, 2018 Supervisor: Dr. W.W.A. Beelaerts van Blokland

(4)

(5)

Abstract

In an era of upcoming trends like Industry 4.0, it is up to the manufacturing businesses to tag along or get left behind. From personal computers to mobile devices, we know that technology can profoundly alter the way we communicate and interact with the world. New technologies impact almost every industry, and logistics is no exception. The next big wave of change in the logistics industry might just come in the form of Augmented Reality (AR) technology. This research assignment aims to explore the possible contribution of AR tech-nology within assembly processes at SEW Eurodrive.

It is important to differ AR from Virtual Reality (VR), whereas AR creates a layer on top of the real world and VR lets the user immerse in another world. The TEL department used its connection with SEW Eurodrive, a manufacturing company, to discuss possible collabo-rations. This resulted in the investment of an AR device, the Microsoft Hololens, and the deal to experiment with this device at SEW in Rotterdam. On a daily basis, a small amount of gearboxes are erroneously assembled. To measure the contribution of AR technology within this assembly process, the objective is to achieve a zero defect assembly due to the Hololens. Research into the hardware and software led to the investment of the AR device and this was followed by setting up an experiment, what later became multiple small experiments to proof the principle behind separate objectives. What follows is a future concept design for further research.

This report concludes with the objective and supporting research questions, showing the achievement of a zero defect assembly might be possible in the future but this could come with less-wanted influences, like time delay. It also elaborates on other elements within the assembly process in which AR could contribute, like training purposes.

(6)

(7)

Preface

To start this report, I would like to tell the student that is continuing this research to carefully read this report and especially the logs that I have included in the appendix. It takes a lot of time to work with all the software and hardware, but when you are at it you will notice the possibilities are endless.

As second I would like to thank Wouter Beelaerts, for guiding me through the process of preparing and executing an experiment. He taught me to perform small steps at a time and sometimes acted as a brake to slow me down, what positively influenced the results of this report.

Last but not least, I want to thank Matthijs Dolman en Niels Maat from SEW Eurodrive. Our conversations first started in September and you showed a lot of patience but also in-terest in my progress since then. The last week acting as a sort of intern was an informative week and being on site really boosted my progress.

G. Epe Delft, February 2018

(8)

(9)

List of Figures

2.1 Helical gearmotor R27/A . . . 3

2.2 Supply inventory work station . . . 4

2.3 Swimlane diagram workstation . . . 5

3.1 Augmented Reality at DHL . . . 8

4.1 Difference inside-out vs. outside-in tracking . . . 12

4.2 Field of View in degrees . . . 12

5.1 Microsoft Academy Tutorial . . . 13

5.2 Vuforia ImageTarget . . . 14

7.1 Experiment set-up . . . 18

7.2 Experiment output . . . 18

7.3 Vuforia database . . . 19

7.4 ImageTarget features [16] . . . 19

7.5 Checklist appears after scanning order . . . 20

7.6 Assembly assistance . . . 20

(12)

(13)

1

Introduction

The manufacturing industry is continuously challenged to innovate their production pro-cesses, while these are becoming more complex, as products become more versatile. The manufacturing processes are required to become more systematic and in a certain way that the production process is conducted in an optimal way. Currently, multiple trends are in-creasing and concurring companies are testing various methods to innovate their production processes.

One interesting method to ease a manufacturing process of a complex product, is the ap-plication of Augmented Reality (AR) technology to determine simulated outcomes of the pro-cess before really executing it. To analyze the influence of this technology, this research is conducted in collaboration with SEW Eurodrive, which is settled in Rotterdam. SEW Euro-drive is a manufacturing company that, for one, assembles gear boxes. On a daily basis, a small number of gearboxes is wrongly assembled and has to be taken apart. To decrease this number to zero, a Microsoft HoloLens is used to integrate AR in the assembly workstation to prevent assemblies to fail.

Transport Engineering and Logistics (TEL) goes further then some harbor sites in Rotter-dam or mathematical logistic models. To innovate the way products are made and attempt a more feasible production process can be seen as a progressive direction within this study. This way of thinking became the corner stone of this research, aiming on zero defect assem-blies. The TEL department has invested in the purchase of AR hardware and aims to achieve multiple students a year to elaborate on this project. Therefore, this paper also applies as a tutorial for future students, so they can catch up in a short period of time and continue the research.

(14)

2 1. Introduction

1.1. Goal of research

The goal of the research is to explore the contribution of Augmented Reality within a manu-facturing process. To optimize the assembly of a gearbox within SEW, the following research objective is formulated:

Can the integration of Augmented Reality technology ensure zero defect assemblies within an assembly process?

This objective question is supported by the following research questions: 1. What is the impact of AR for the employee in relation to

(a) real time information during assembly? (b) movements of parts and employee handling?

(c) the process time?

(d) the rework of a defect assembly?

2. How is AR technology perceived during the experiment in relation to achieving zero defects rework assembly?

3. Which hard- and software are used to do the experiment?

The answers to these research questions will be assembled to give a solution for the research objective.

1.2. Research structure

To clearly answer the research questions, this report is divided in multiple chapters. At first, an insight in the concerning company SEW will be given. A close look at the company and the current state of the assembly process will be provided. This is followed by the AR technology and its software and hardware.

Chapter six will elaborate on the future potential of AR technology integrated within the assembly process at SEW and this will be supported with a SWIMLANE process flow. The following chapter contains the experiment with Augmented Reality at SEW, with the com-plete set up and all necessary data. The report is concluded with a future concept design with recommendations.

With self-study and experimenting with the Microsoft HoloLens and its software, the the-ory part of this report will be executed at first. The goal is to set up multiple plans for the experiment, to end this research with the experiment while being prepared for unexpected drawbacks. Because this research is meant to be continued, a well laid plan for the experi-ment is the least this report should provide for further research.

(15)

2

SEW Eurodrive

SEW Eurodrive is a German manufacturing company, internationally known for manufactur-ing geared motors, frequency inverters and electrical drives. Earlier contact with employees from the plant in Rotterdam has led to a collaboration, resulting in a lot of data provided by SEW Rotterdam.

One of the key elements within SEW Rotterdam is the integration of the Lean Six Sigma method, which aims at value-added steps in the production process and cutting the non-value added steps. It can be said that SEW Rotterdam is a progressive plant with a good focus on Lean production.

The manufacturing industry is constantly facing the challenge of producing innovative prod-ucts at reduced production time. Like every factory, there are elements that can be improved within the production process of SEW Rotterdam. One of them is the defect assemblies, which amount lies around five each day. An incorrectly assembled assembly must be taken apart and build over again. This is one big series of non-value-added steps. The goal of this research assignment is to achieve a zero defect assembly, using Augmented Reality technol-ogy.

2.1. The assembly

The product that will be assembled for this research is called the R27A and is a helical gearmotor, as can be seen in figure 2.1. An employee of SEW Rotterdam is trained to assemble these kind of gearboxes all day but the plant can never guarantee hundred percent correct assemblies. To avoid these so-called defect assemblies, this research is looking into the addition of an AR Technology glasses in this manufacturing process.

Figure 2.1: Helical gearmotor R27/A [3]

(16)

4 2. SEW Eurodrive

2.2. Current state of process

The assembly process of the gearbox finds place in a work station, like a little island. The employees can move freely within this island while the assembly starts at one point. Prior to the process, the employee receives a production order document that states what the char-acteristics are of the required assembly, this document is then linked to a bill of material. This bill of material shows a list of all parts that are needed for the assembly. This document functions as a guide to direct the employee in the correct direction. The employee has a workbench with integrated press to ensure some parts are fitting right in. Appendix A shows a document with detailed assembly instructions that show that some manufacturing steps include adjusting very small parts, these instructions are spread around the work station and can be used at all time during the assembly. All work that is done can be seen as blue collar labor that mostly use their hands and eyes and due to the many repetitive tasks.

All the parts that are necessary for the assembly are provided in boxes, as can be seen in fig-ure 2.2. The man responsible for the fabrication only looks at the picking document once to know already half of the steps he has to take. With the help of the lever press, he assembles the whole housing and he moves it once to another press where he or a colleague finishes the assembly. Because all available employees are well-known within these processes, there is not a specific part standing out, acting as a bottleneck. At least, within this assembly pro-cess. However, like in every company, new employees need training to achieve the skills that the older colleagues possess as well and this may cause an extra dimension to the objective.

The objective, to ensure a zero defect assembly, is stated as a problem that is in need of a solution. Currently, the amount of erroneously assembled products can be counted on one hand. When an assembly has to be taken apart, it is sent to another work station. Here the employees don’t need input data or a guide what to do, they know the assembly has to be taken apart and with some tools they just apply forces where needed and disassemble the product. The time needed for this action is not constant and therefore AR has no impact worth mentioning.

(17)

2.3. Swimlane Diagram 5

2.3. Swimlane Diagram

To illustrate the current state of the process within the working station, figure 2.3 will provide a Swimlane diagram to clarify the input that is necessary for the employee to obtain before starting the assembly. The focus of the assembly process is the input data that is obtained by the employee and how he handles the data.

(18)

(19)

3

Augmented Reality Technology

To define Augmented Reality, it is expected to relate the differences with Virtual Reality (VR). Whereas VR glasses let the user give the feeling that they are ’inside’ a virtual world, the level of immersion differs with AR technology, where a virtual world is implemented in the real world.

AR simply overlays computer-generated information on the real world environment [7]. It uses a build-in camera to derive presented information from the real environment, which means it is sensitive to the context. In this way, AR enhances the existing environment, in-stead of replacing it like VR does. It can let the user’s view fill with virtual information that is sensitive to the current state of the surroundings, which leads to the integration of AR within manufacturing processes.

3.1. The technology

The technology behind AR can be divided in the following subjects, that will be explained shortly after:

• Tracking • Display • Development

• Input and interaction

The tracking depends on the built-in cameras that the AR glasses contains. There are mul-tiple ways of tracking, with magnetic fields; based on visible light but also based on placed markers in the room the user is located. The application of GPS tracking was rejected right away, just like the magnetic trackers for it would be to inconsistent. The right candidate in the field of this research were the AR devices that made use of 3D structure tracking, for the built-in sensors utilize structured light to obtain information about three dimensional positions of points in the area, without using markers.

The biggest difference between VR and AR devices is the display. Where VR lets the user immerse in its virtual surroundings, the display of the AR would create a virtual layer on top of the physical world around the user. Where VR attractive for gamers, the technology of AR is more likely to be used within manufacturing companies. Chapter four will elaborate further about the display.

(20)

8 3. Augmented Reality Technology

The development takes place in various ways. There are possibilities to let go of the cre-ative mind and create a tremendous virtual world in Unity, well-known by gamers. All this creativeness has to be connected with each other and when that part arrives, so does the pro-gramming side of the development. Basic propro-gramming skills are recommended, depending on which software you are using. In the case of Unity, C-sharp is the programming language that is used. More about the software used during this research returns in chapter five.

This research sees the fourth subject as the main priority; what is the input and how does the user interact with its surroundings? The most AR devices are actually small computers that are able to create a virtual world, sometimes without the help of actual computers. For example, the Microsoft Hololens has a Windows Start menu and can let Internet browser windows appear or let the user play videogames, downloaded from the app store. This with-out interference of a personal computer. Furthermore abwith-out the input will be discussed in the software section.

The interaction of the user is one of the biggest innovation of the past decade. With help of sensors and cameras, the user is able to use his head and hands to interact with the AR device. Where his head can be used to move a cursor, its hand can (with or without motion controllers) interact with holograms and windows that appear on screen. The device measures the depth between the surroundings and between the fingers of the user and can therefore capture the movements.

3.2. Current state

AR can be seen as a promising technology that in the near future will be integrated in every-day products. With known brands like Apple and Microsoft, that are also aiming on AR technology within their products, people get more familiar with AR which can lead to an eas-ier acceptance if the technology is integrated within jobs.

Currently, AR technology is being used at certain manufacturing companies. Order pick-ers at DHL wear an AR glasses whilst driving their forklift trucks through the distribution halls, while their screen shows where they need to go, as can be seen in figure 3.1. Every-day smartphone apps and websites are already using it, think about Snapchat filters or the summer-hype of Pokemon GO.

In the medical world and education it is a trend to implement AR technology in their train-ing. Someone could get a stroke on the street and someone on site can scan the body of the wounded person and be in contact with a doctor right away, so the doctor can support the helping passenger. In another way, lectures about anatomy can be educated more accurate than ever [5].

(21)

3.3. Future state 9

3.3. Future state

The medical world and education are not the only sectors that benefit from AR integration. In the same way, AR can be used for training in the industry sector. Especially where manual labor must be conducted, it could cost time to let someone tell the new employee what to do and how to do it. This happened to be one of the things that the employees at SEW were interested about as well.

(22)

(23)

4

Augmented Reality Hardware

This chapter will provide the reader with all required knowledge about the AR hardware that is purchased; the Microsoft HoloLens. This chapter will start with an overview of the characteristics of common AR glasses and which had priority during this research [2, 12]. Appendix B will contain a proposal that is written for the TEL department to invest in the Hololens. This section will discuss the third research question, investigating the hardware.

4.1. Requirements

The main angle of this report is to research the contribution of AR in a production process. To do so, the investment in hardware is required. AR hardware can be characterized by the following points, which will be explained as well:

• Wireless

• Wi-Fi / Bluetooth

• Markerless inside-out tracking • Field of View

• Battery life

4.1.1. Wireless

To test the AR glasses, the hardware needs to be able to move freely without thinking about cables. This only can cause the user to fall or incidentally influence the surroundings in an unpleasant way.

4.1.2. Wi-Fi / Bluetooth

This property connects narrowly with the previous one. To program the software, the hard-ware must be able to connect with a computer to access the Microsoft Mixed Reality softhard-ware, more about this in the next chapter.

4.1.3. Markerless inside-out tracking

An essential component of AR, contributing to a adequate sense of immersion and presence, is positional tracking. This determines the position and orientation of an object within the environment, which is necessary for placing digital content into a real environment [9].

The addition of markerless, inside-out tracking can be seen as a method of motion track-ing without the use of markers. It is common to place certain markers within the users view to support the system indicating the position of the user/camera. The markerless method uses objects that are already present in the surroundings and therefore is more flexible to use.

(24)

12 4. Augmented Reality Hardware

The meaning of inside-out tracking is made clear in figure 4.1. With inside-out tracking, the user is not dependent on cameras in the environment and can maneuver freely.

Figure 4.1: Difference inside-out vs. outside-in tracking [6]

4.1.4. Field of View

A property that is seen as one of the most important one, is the Field of View (FoV). This property is expressed in degrees and indicates the users view while using the AR hardware. For users at SEW a high FoV is not necessarily required, for the user is able to rotate its head slightly to see all available parts.

Figure 4.2: Field of View in degrees [8]

4.1.5. Battery Life

Like with every technical product, the battery life is a recurring obstacle. Competitive com-panies always claim to have improved the battery life of their product with the newest tech-nologies. The focus of this research lies in creating a software for employees at SEW, to help them assemble an electro motor. In this phase of the research, a long lasting battery life is not a priority. However, it is expected that future products can last the required eight hours that cover a normal working day in the Netherlands.

4.2. Investment in Hololens

Due to the earlier mentioned characteristics, a research could be performed to all available AR devices, resulting in a clear overview that can be found in Appendix C [11, 15, 18]. After the overview was completed, a choice and a proposal was made for an investment in the Microsoft Hololens. In the end, the markerless inside-out tracking and the fact that the device is made by Microsoft were decisive when choosing.

(25)

5

Augmented Reality Software

Windows Mixed Reality is the comprehensive name for Windows apps and tools that are used to work with AR and VR technology. This chapter will mainly discuss the three most impor-tant software programs that come in play while working the Hololens: Unity, Vuforia and Microsoft Visual Studio. This section will discuss the third research question, investigating the software.

One of the arguments for choosing the HoloLens is the Mixed Reality Academy that Microsoft offers [10]. This Microsoft Academy stimulates developers, creators and students to build mixed realities and therefore offers a decent amount of tutorials. This enables students without programming experience to become an expert in a relative short period. This trend is continued within the communities of Unity and Vuforia, where a lot of programmers share their creations and help each other out when one is stuck.

5.1. Unity

The first software program is the one that is mostly used and particularly known by gamers, because it is used to create video games for Nintendo, Playstation and for the Xbox. It can be used to create environments, 2D and 3D objects, but also to activate the camera on the Hololens [13]. Figure 5.1 shows a project that is made with a tutorial from the Academy, in which spatial mapping is applied. This causes the Hololens to map the whole surroundings and sets boundaries, like walls, the floor and the ceiling. In this case, a hologram can be put on the floor instead of falling through.

Figure 5.1: Microsoft Academy Tutorial [10]

(26)

14 5. Augmented Reality Software

5.2. Vuforia

Vuforia can best be seen as a glorified add-on, that in the newer versions is integrated within Unity. There are a few tasks that make the use of Vuforia quite interesting, with as first the ’ImageTarget’ function. This function let the user upload and print a picture at first and thereafter connects an object, which could be a text or a hologram, with this picture like in figure 5.2. An other advantage of using Vuforia is the additional database that can be created. At the Vuforia developers website, the user is able to create an account and obtain a license key and this key can be entered in Unity. This gives Unity access to the online database. This database will return in chapter seven.

Figure 5.2: Vuforia ImageTarget [17]

5.3. Microsoft Visual Studio

This software program is the least used, but therefore not the least important. It is mostly used for the Hololens for two tasks: to write programming language (in the case of Unity in C-sharp) and to deploy the app that is created in Unity. The former is not used in this research but will be discussed in the recommendations. The latter can be used to convert the created scenes in Unity into an app that is downloaded to the Hololens, so the app can be used without interference of the desktop.

5.4. Hololens compatibility

It is very important that the student that follows this research, visits the Vuforia development forum to check which versions of Unity and Vuforia are recommended to use. At the time of writing, the newest versions of Unity and Vuforia are indicated as not being compatible with the Hololens, however, in contact with AR software companies in Rotterdam it was concluded that it is possible to work with both the Hololens as with the newest software.

(27)

6

Integration AR Technology at SEW

Theoretically, AR can be of great impact in the manufacturing industry. With an extra layer of data over the real world, it almost can not be seen as a disadvantage. Still, the level of impact depends a lot on the dimensions of the process. This section will partly answer the first research question, investigating the impact of AR.

6.1. Current employees in plant

In discussion with employees on the work floor, it became clear that a relatively small plant as SEW Rotterdam is not in desperate need of AR technology within the assembly process. The objective on site was to look at possible tricks that could be enhanced or upcoming bottlenecks that could be solved with the Hololens. The conclusion at the R-27 assembly station stated that the contribution of the Hololens would take a lot more time than it would save. Personnel being active for a longer period can see the order document and in one blink already know the firs ten steps they have to execute. If they have to register every small part they take for the assembly, this would cost more time than necessary.

6.2. Mechanics on location

In contrast to employees on site, there are also mechanics 24/7 available to perform mainte-nance on location. This does not contain a repetitive task, executed a lot of times a day and therefore the contribution of AR could be of impact.

6.3. Training and temporary personnel

A company undergoes changes to keep up with the trends. Processes innovate, company structures adjust and inevitable, personnel gets substituted. Here is an aspect where the AR technology could contribute, to ensure new personnel gets the proper training without taking time of other personnel. When an employee calls in sick and temporary manpower is deployed, he also can deliver a proper result.

(28)

(29)

7

The Experiment

The final part of this research included a full-time presence at SEW Eurodrive to execute an experiment. The preparation of this experiment took place in a basement at the TU, where the basics were mastered and where literature study contributed to setting up the experiment [4] [1] The objective of this research is to make steps in the direction of a zero defect assembly. This section will elaborate on the first and second research question, the impact of AR and how AR is perceived during the experiment.

7.1. Original plan

The original plan can be explained on the hand of three elements;

1. The set-up

2. Input data

3. Output and handling

7.1.1. The set-up

The set-up can be explained best with support of figure 7.1, which shows the workbench on which the employee assembles the product. At the edge of the workbench, an inventory is available with all the parts that are necessary for the assembly. In three steps, the user will pick the right part for the assembly:

1. The picking order list contains a code, that is linked to a checklist.

2. The codes linked to all available parts are scanned as well and this is connected to the checklist, so the checklist knows if all parts are even present.

3. When all needed components are available, the checklist will commence and light up the box containing the first item on the checklist. This will continue until the checklist is fully checked.

7.1.2. Input data

There is less input data needed then thought upfront. With only one picking order, all needed information about the dimensions, components and manufacturing method is provided. Sev-eral picking orders can be linked to sevSev-eral checking lists and these checking lists can all be saved in the Vuforia database. The other input data that is required has an equal method as the first one; the codes that link the parts to the checklist. The software knows that the code is presented a few centimeters below the center of the part, so the software also knows which area needs lightening.

(30)

18 7. The Experiment

Figure 7.1: Experiment set-up

7.1.3. Output and handling

The set-up and input data promise a process that can run without errors, the Hololens’ output ensures the expected outcome. Figure 7.2 shows the outcome of the experiment, it represents the view of the user and is also explained via a few steps.

1. When scanning the picking order, the corresponding checklist appears on screen.

2. All the parts in the inventory are being scanned and whilst scanning, they get a color in the list indicating they are present.

3. When the assembling is started, this step shows from the top of the list to bottom which part is to be used, indicated by a small glow around the box.

4. When two following parts are taken, a small video shows how these two parts are meant to be assembled into one part.

(31)

7.2. Alternative Experiment 19

7.2. Alternative Experiment

It became clear that this objective is realistic in the near future and that for the current re-search, the fundamental steps have to be tested. Therefore, another experiment was created; a majority of smaller, more achievable experiments. The main focus within these experi-ments will be conducting the Proof-of-Principle: with the main objective in mind, the smaller experiments will show the feasibility of the project. Every experiment also contains a small recommendation for the successive student that continues this research.

7.2.1. Experiment 1: Scanning components

This experiment starts with the most essential part of the objecive: enable the Hololens to scan images and link these images to the database. Using the Vuforia website, a visual database is created as can be seen in figure 7.3. The database is then imported within Unity and can be linked to the outcome that is wanted by the user.

Figure 7.3: Vuforia database [16]

Figure 7.4: ImageTarget features [16]

During this research, the active presence of Vuforia within Unity caused the software to crash or improper operate when connecting to the Hololens. This finally led to the decision to use a laptop with webcam, so at least the principle could be proven. Figure 7.4 shows the way the database registers pictures and codes, based on contrast based features.

(32)

20 7. The Experiment

7.2.2. Experiment 2: Checklist

As an outcome of the first experiment, the second goal is to let a checklist appear on screen. As mentioned above, the employee receives a picking list at the start of the assembly process. In the experiment, this list is scanned and connected to a database in Vuforia. The recogni-tion of this list executes a checklist on the Hololens screen and, as can be seen in figure 7.5, illustrates all parts needed for the assembly.

Figure 7.5: Checklist appears after scanning order

This research managed to create a single checklist that stays on screen, even after the order paper is removed from screen. This action can be repeated with all possible checklists and so this principle is proofed to be operational.

7.2.3. Experiment 3: Assembly assistance

There are multiple ways to decrease the amount of wrong assemblies. Instead of making it easier not to pick the wrong part for the assembly, the right way of conducting the assembly can also be focused on. It is possible to execute a small movie that functions as assembly assistance, after two following parts of the checklist are obtained. This will appear as a small window and can show an employee that assemblies the two parts in a cautious and clear way. Figure 7.6 shows this method.

(33)

7.3. Trial and error 21

7.3. Trial and error

The appendix of this report consists of multiple documents. Appendix D should be read instantly after reading this report, to help the student kick-starting his or her research. It describes a roadmap with the recommended steps anyone needs to follow when starting working with the Microsoft Hololens and the corresponding software.

(34)

(35)

8

Future Concept Design

This chapter concludes the research and this report with a future concept design. The Ap-pendix contains a roadmap and other tips and tricks that can serve as a manual when work-ing with the hardware and software. After controllwork-ing both, this chapter can be relied on to continue the research. The concept that will be elaborated on is the same as the original experiment, but explained separately and with the recommended software.

8.1. Software and Hardware

The first steps should be checking the hardware and software compatibility. Vuforia develop-ers tell in the forums on their own site that the compatibility of the Hololens with the software is not supported yet, however, Dutch companies specialized in AR technology claim other-wise. Recommended is to contact several companies and ask them what the possibilities are.

8.1.1. Tutorials

In the roadmap enlisted in Appendix D, a few links with tutorials are given. These tutorials give a clear view of the endless opportunities the software can provide. Save this tutorials, try the basic ones and know to fall back on these if things do not work out. Often, there is a bit of theory to be found in the tutorials and also a lot of tips and tricks are provided.

8.1.2. Windows PC

The roadmap also gives a link to the Microsoft homepage. This page indicates what kind of configurations the Windows computer must run on. The workstation provided by the TEL department is already accepted but to check for updates sometimes is recommended.

8.2. Input and connection

The Vuforia database is during this research only used for 2D Image Targetting. This database can be expanded with 3D objects as well, which makes it more attractive to scan the compo-nents itself instead of codes positioned underneath the parts. So the input is already proven in section 7.2.1, if this action is done repetitively, a database is created with all possible orders. The following step is to make the checklist interactive.

At this point, the checklist is a image that moves with the user on screen, but is not able to change. When the list is on screen and components are being scanned, the checklist should react on the object to illustrate the parts are available. When all the parts are indicated as present, the user should be able to push a ’Start’ button or to say ’Start Assembly’ and the checklist should react on this signal.

(36)

24 8. Future Concept Design

An interesting input could be the mapping of the raster in which the parts are presented. To scan all the boxes with parts and codes could lead to the Hololens to know exactly where all the parts are located, this can be used to highlight the boxes more easily at the output.

8.3. Output

The output of the checklist is followed by the input signal to start the assembly. The checklist should be able to highlight the first line on the list, followed by the highlighting of the box. In an ideally situation, when two following parts are taken from the inventory and scanned, a movie or two holograms of the parts appear on screen to assist the employee during the assembly of the two parts. During this research, this part worked with a webcam and 2D images, there is a promising future state using the Hololens and 3D objects.

(37)

9

Conclusion

To finalize the executed research, this concluding chapter will discuss the objective and re-search questions that are introduced in the first chapter. The process of investing in an AR device and trying to program it was a stubborn but informative one. The interest of the TEL department already lay within AR and VR technology and when the contact with SEW increased, the objective around an assembly process was made up. This chapter will discuss the results of the objective.

Can the integration of Augmented Reality technology ensure zero defect assemblies within an assembly process?

To answer this research question, a few supportive questions were answered during the project:

1. What is the impact of AR for the employee in relation to (a) real time information during assembly?

All the information that the employee receives comes from a single production order document. Currently, the employee is expected to recognize all the component codes and link them to the physical parts. The impact of AR is shown in chapter seven, where the order document can be scanned and the checklist on screen shows which components are required and in what order they have to fit in the assembly. This virtual checklist can differ and is linked to a database where various lists are linked to different production orders and when this database is updated, the virtual checklist on screen is also updated.

(b) movements of parts and employee handling?

This research does not focus on the supply of the inventory but on the handling of the parts. The employee sees through the AR glasses what component he has to take from the checklist and the glasses also indicates the position of the item, so the impact is that the employee has more certainty to pick the right part and in the correct order.

(c) the process time?

The impact of AR for the employee in relation to the process time can be seen as both positive and negative. It can be negative for the employee that is well known with the assembly and gets slowed down, a positive impact is delivered to someone who does not know how the process works (yet) and can learn from the device. (d) the rework of a defect assembly?

The impact of AR within the rework of a defect assembly was not measured during this research after it was clarified there is no standard procedure for this process. The employee analyses the erroneously assembled product and uses force where needed to disassemble it.

(38)

26 9. Conclusion

2. How is AR technology perceived during the experiment in relation to achieving

zero defects rework assembly?

The employee uses the AR glasses to scan a production order that as a result shows a virtual checklist on the inside of the glasses. When scanning the available components, this checklist indicates if all parts necessary are present. When starting the assembly, the glasses show a light or glow around the inventory box where the required part is located. After picking two following parts, a screen appears with a movie that provides assembly assistance.

3. Which hard- and software are used to do the experiment?

This research started with investigating all available hardware. After sorting out which factors were of importance and which were negligible, an investment proposal was made to purchase a Microsoft Hololens. To operate this device, there are multiple software programs utilized. The program Unity is used as the base program to create scenes, these scenes are filled with objects and shows the user what he can see with the AR glasses. Vuforia is an add-on for Unity which deliveres a database and handy features, like linking real images to virtual objects. Microsoft Visual Studio is used to edit the programming code and to convert the scenes created in Unity into an application and deploys this app in the Hololens.

These subquestions were supportive and showed the impact of AR in multiple ways, not only with the focus on a zero defect assembly, but it also showed possibilities to do research into other directions. It is not possible to answer the objective question with supportive tests with results, but there is a good possibility that the integration of Augmented Reality technology can ensure zero defect assemblies within an assembly process. However, at what cost? For now it was not measurable, but it is expected that the integration within the assembly process will cause a certain delay that could be more influence than the AR technology.

This research started with the idea to invest in an AR device and create an objective that was executable within SEW. When starting with the Hololens, it became clear the objective was a good direction but was not going to be completed within this research. With this di-rection in mind the experiment was altered into smaller experiments, aiming on results that could be fundamental for further research.

This report can be used as a basis and tutorial for further research. The appendix con-tains important documents that are recommended to consult and will kick-start the next student who will wear the Hololens.

(39)

A

Manual with assembly instructions

(40)

Assembly Instructions

01 793 02 99

Helical gear units _EN _{Page 1/4}

R/RF27 _LMI _14.03.08

Pinion shaft installation, 2-stage version

Für diese technischen Unterlagen behalten wir uns alle Rechte vor / Copyright reserved / Tous droits de modification réservés SEW-EURODRIVE GmbH & Co KG Postfach 3023 . D-76642 Bruchsal Tel. ( 07251 ) 75-0. Fax ( 07251 ) 75-1970 . http://www.sew.de

This assembly instruction is matched to the assembly tool kit "Complete".

Individual steps are explained with tools that are not included in the assembly tool kit

"Productivity" or "Quality-relevant".

Suitable tools have to be used in this case.

Quality-relevant tools are marked with an X in the "Q" column in the relevant tool parts list.

Pre-assemble pinion shaft 5.

Parts :

-

Gear unit housing 22

-

Circlip 33

-

Bearing 34

-

Pinion shaft 5

-

Spacer tube 32

-

Key 31

-

Gear 2

Tools and material :

-

Press-in support W3

-

Support plate W30/1

-

Lever press

1 Axial

back-up

ring

Only with i=10,13 up to 28,37:

Install circlip 33 on pinion shaft 5.

2 Place bearing 34 on W3,

(Loose bordring always on top !)

Remark

:

The bearing must be handled carefully to avoid

falling apart of the three pieces (inner ring, outer

ring and loose bordring)

3 Join pinion shaft and press in with lever press.

Caution

:

Inner ring and outer ring are paired

( the interchange of these parts is not permitted)

Orientation: loose bord must point to the toothing

of the pinion always

(41)

Assembly Instructions

01 793 02 99

R/RF27 _LMI _14.03.08

4 Slide spacer tube 32 on pinion shaft,

install key 31 in pinion shaft 5.

5 Place gear unit housing 22 on support plate

W30/1.

Support element for output shaft and centering

tip are lowered.

(42)

Assembly Instructions

01 793 02 99

R/RF27 _LMI _14.03.08

Press in pinion shaft 5 and bearing 37.

Parts :

-

Circlip 35, 38, 39

-

Bearing 37

Tools and material :

-

Pressing tool W6

-

Support plate W30/2

7 Slide support plate W30/2 under gear 2.

8 Position pinion shaft 5 in gear 2.

9 Place circlip 38 in bearing bore for bearing 37.

10 Place bearing 37 on pinion shaft 5. Press in

entire assembly with pressing tool W6.

(43)

Assembly Instructions

01 793 02 99

R/RF27 _LMI _14.03.08

11 Remove support plate W30/2.

12 Press in pinion shaft 5 with W6 until circlip 38

snaps in place.

(44)

(45)

B

Investment proposal Microsoft Hololens

(46)

Gerco Epe, student Transport Engineering and Logistics

PROPOSAL FOR INVESTMENT

AUGMENTED-REALITY GLASSES

RESEARCH ASSIGNMENT

To research the possibilities of Augmented Reality by an experiment within a production process.

To perform the experiment a device so called “Hololens” is selected. The motivation for this device

is described in the next sections.

A. Microsoft HoloLens (hardware)

i.

Multiple Augmented and Virtual Reality devices were analysed, the HoloLens was chosen to

be the best solution in line with the requirement for the experiment.

ii.

The focus with the HoloLens stays on AR. However, the dimensions of the devices ensure

there are VR possibilities.

iii.

The Field of View (FoV) regarding the device reaches 40 degrees, which is sufficient for our

project. If parts are not in reach within the FoV, augmented arrows can indicate the way the

user needs to move.

B. Windows Mixed Reality (software)

i.

The software that comes with the HoloLens is compatible with multiple variants of

computer-hardware and software. With all computers that are stationed in the faculty

running Windows, this software won’t fail.

ii.

Microsoft offers a large number of tutorials and similar self-study tutorials to ensure the

user to be able to develop the required programming skills on itself. This also makes the

device accessible for other students in a later stage. Windows differs from other solutions

especially on “tutorials”. This is a necessary aspect to prepare for the experiments, which

other suppliers do not offer in the extend of Windows Hololens.

iii.

With Windows Mixed Reality software, the Virtual Reality aspect of the HoloLens is

incorporated. The immersive feeling that VR-users expect can also be experienced with the

HoloLens. This makes this device usable in multiple projects within the faculty AR-VR LAB.

iv.

Visualization of the software and hardware:

https://www.youtube.com/watch?v=2MqGrF6JaOM

C. Price indication

i.

The HoloLens is available for $3,000, which calculates to €2,500. This ‘Development Edition’

comes with all kinds of documentation, tutorials and community forums for the individual

developer to start with.

ii.

The HoloLens is delivered with all hard- and software that is necessary to experiment with

the device.

(47)

C

AR device analysis

(48)

Company Model Price Wireless Wi-Fi / Bluetooth Software Markerless inside-out tracking Field of View

[degrees] Battery Life Google Glass € 1.300 Yes Yes Glass OS No

ODG 'Osterhout Design Group' R7 - industrial use € 2.500 Yes Yes ReticleOS No 30 4.5 hrs 3D movies R8 € 850 Yes Yes ReticleOS No 40 4-5 hrs 3D movies R9 (Fall 2017) € 1.500 Yes Yes ReticleOS No 50 5.5 hrs 3D movies Microsoft Hololens € 2.500 Yes Yes

Windows

Mixed Reality Yes 40 2-3 hrs active use

Vuzix M100 € 700 Yes Yes Android No 15 6 hrs

M300 € 1.500 Yes Yes Android No 17 2-12 hrs depending type of use Sony SmartEyeGlass € 700 Yes Yes Android No 20 irregular'

MetaVision Meta 2 € 1.300 Yes Yes unknown Yes 90 unknown Magic Leap not yet released € 1.700

Telepathy One No

Walker No

Jumper Android No

Lumus Maximus not yet released No 55

Requried for SEW / TU Delft ?? Yes Yes n.v.t. ?? ?? at least a few hours for research No VR/AR compatibility

VR/AR compatibility Uncertain compatibility

(49)

D

Roadmap for Microsoft Hololens

(50)

Roadmap Microsoft HoloLens

Dear student,

This document will contain some steps that I recommend you to follow when starting

working the Hololens. Like mentioned in the report, multiple software is being used in the

process. If you follow the five steps below, you will know what the software means and does

within a day.

1. Read the round manual about the Hololens

2. Create an account on Microsoft.com

3. Put on the glasses, activate it and follow the tutorial.

4. Go to

http://developer.microsoft.com/en-us/windows/mixed-reality/install_the_tools

and install all the tools necessary (probably already done on

the system you are working) Watch out: the newest version of Vuforia is already

integrated in the newest Unity.

5. Start following the tutorials on

https://developer.microsoft.com/en-us/windows/mixed-reality/academy

After these 5 steps you will have a pretty good idea about the possibilities, from

programming in Visual Studio as being creative in Unity. Below are more tutorials and tips

that will guide you through the basics. There is also an asset store in Unity, I recommend

downloading some Vuforia Samples to check if your software is well installed.

When you import samples or assets, you will have to look for SCENE files. If a map contains

scene 0,1 and 2, you’ll have to put all of them in the project window. These scenes are linked

to each other and need to be played at once.

https://docs.unity3d.com/Manual/index.html

https://unity3d.com/learn/tutorials

https://library.vuforia.com/articles/Training/Developing-Vuforia-Apps-for-HoloLens

Creating an app

When creating an app, think about some standard settings for the Hololens. You will have to

install Windows 10 SDK, which contains standard packages for Visual Studio. The rest you

can find on

https://developer.microsoft.com/en-us/windows/mixed-reality/performance_recommendations_for_unity

My experience:

After a lot of trial and error, it appeared I was watching old tutorials. These tutorials let me

integrate OLD Vuforia packages while the newest version was already integrated. If you are

stuck, call with AR companies (there is TWNKLS in Rotterdam) for tips, this can prevent you

from spending too much time on nothing.

Finally, I was not able to use Vuforia in combination with the Hololens, that is why I did a

part of my experiment on my Macbook.

This should be one of the first steps for you to achieve!

Succes!

(51)

E

Research timeline

(52)

Timeline Research Assignment Gerco Epe

26 September – Meeting with Mr. Beelaerts and SEW Eurodrive, discussing the possibilities

of VR and AR integration within production processes.

9 October – Aligning research objective with Mr. Beelaerts, in this period with the main

focus on VR instead of AR.

23 October – Proposal for investment Microsoft Hololens is submitted.

25 October – Meeting with SEW, agreement to focus on AR and planning on experimenting

with AR device at SEW.

13 November – Planned date to start experimenting with Hololens.

8 January – The AR device came in, so experimenting could start.

It was discussed with SEW that in the week of 5 – 9 February, I would be located in the office

to experiment with the Hololens. What follows is a summary of the achievements of this

week:

5 February – Started at SEW, continued trying to let Vuforia operate with Hololens. Found

out that multiple scenes can be added to the project.

6 Feruary – Read on Vuforia developers site that Vuforia and Unity versions were not

compatible with Hololens, so older versions of the software had to be installed. After

installation, still no success.

7 February – Vuforia is the quickest and easiest way to achieve a link between code and

data, but due to failure within Hololens the software is put aside for now. A checklist is

created and programmed so that it stays on screen while moving around.

The idea of using the Macbooks webcam came across, but the focus of this research lies

within the Hololens so therefore the choice was not to go for it.

8 February – A video conversation was made with the headquarters in Bruchsal, Germany.

Discussion about the AR device and possibilities resulted in obvious conclusions, they appear

to outsource all the programming tasks to an AR company. I had a conversation with a local

AR company to ask about Vuforia tips, but no result came from that.

This day the earlier choice not to use the webcame was reversed and as a result, the Vuforia

add-on could be used to proof the principles behind the small experiments. This ensured

that this research had executed more experiments than expected.

9 February – The last day consisted of meeting with Mr. Beelaerts and SEW, discussing the

results and further research.

(53)

Bibliography

[1] N. Imbert F. Vignat C. Kaewrat P. Boonbrahm. Adding physical properties to 3d models in augmented reality for realistic interactions experiments. International Conference on Virtual and Augmented Reality in Education, 2013.

[2] M. Con. The best augmented reality smart glasses 2017. http://havethisbeast.com/best-augmented-reality-smart-glasses-2017/, (Online), 2017.

[3] SEW Eurodrive. Standard gear units. https://sew-eurodrive.nl, (Online).

[4] C. Patron G. Reinhart. Integrating augmented reality in the assembly domain - funda-mentals, benefits and applications. Institute for Machine Tools and Industrial Manage-ment, 2003.

[5] S. Gardonio. The current state of augmented and virtual reality. https://www.iotforall.com/current-state-augmented-virtual-reality/, (Online), 2017. [6] Hirotake Ishii. Augmented reality: Fundamentals and nuclear related applications. 12

2010.

[7] B. Jones. Before augmented reality becomes the next big thing, here’s what needs to happen. https://www.digitaltrends.com/virtual-reality/augmented-reality-gesture-controls/, (Online), 2017.

[8] O. Kreylos. Hololens and field of view in augmented reality. http://doc-ok.org/?p=1274, (Online), 2015.

[9] B. Lang. Hololens inside-out tracking is game changing for ar and vr, and no one is talking about it. https://www.roadtovr.com/microsoft-hololens-inside-out-tracking-augmented-reality-virtual-reality-ar-vr/, (Online), 2016.

[10] Microsoft. Microsoft windows mixed reality academy.

https://developer.microsoft.com/en-us/windows/mixed-reality/academy, (Online), 2013.

[11] ODG Osterhout. Smartglasses. https://www.osterhoutgroup.com/, (Online).

[12] Smartglasseskopen.com. Ar brillen. http://www.smartglasseskopen.com/ar-brillen/, (Online), 2017.

[13] Unity. Learn with unity. https://unity3d.com/learn, (Online).

[14] G. van Marle. Dhl supply chain eyes new augmented reality as it expands use of smart glasses. https://theloadstar.co.uk/dhl-supply-chain-augmented-reality/, (Online), 2016. [15] Meta Vision. Meta 2. https://meta-eu.myshopify.com/, (Online).

[16] Vuforia. Target manager. https://developer.vuforia.com/target-manager, (Online).

[17] Vuforia. Asset store unity. https://assetstore.unity.com/packages/templates/packs/vuforia-core-samples-99026, (Online), 2017.

[18] Vuzix. Smart glasses. https://www.vuzix.com/, (Online).

The contribution of Augmented Reality to a zero defect assembly

The contribution

of

Augmented

Reality to a zero

defect assembly

G. Epe

Research to the integration of AR technology within

assembly processes at SEW Eurodrive

The contribution

of Augmented

Reality to a zero

defect assembly

G. Epe

Abstract

Preface

Contents

List of Figures

1

Introduction

1.1. Goal of research

1.2. Research structure

2

SEW Eurodrive

2.1. The assembly

2.2. Current state of process

2.3. Swimlane Diagram

3

Augmented Reality Technology

3.1. The technology

3.2. Current state

3.3. Future state

4

Augmented Reality Hardware

4.1. Requirements

4.1.1. Wireless

4.1.2. Wi-Fi / Bluetooth

4.1.3. Markerless inside-out tracking

4.1.4. Field of View

4.1.5. Battery Life

4.2. Investment in Hololens

5

Augmented Reality Software

5.1. Unity

5.2. Vuforia

5.3. Microsoft Visual Studio

5.4. Hololens compatibility

6

Integration AR Technology at SEW

6.1. Current employees in plant

6.2. Mechanics on location

6.3. Training and temporary personnel

7

The Experiment

7.1. Original plan

7.1.1. The set-up

7.1.2. Input data

7.1.3. Output and handling

7.2. Alternative Experiment

7.2.1. Experiment 1: Scanning components

7.2.2. Experiment 2: Checklist

7.2.3. Experiment 3: Assembly assistance

7.3. Trial and error

8

Future Concept Design

8.1. Software and Hardware

8.1.1. Tutorials

8.1.2. Windows PC

8.2. Input and connection

8.3. Output

9

Conclusion

A

Manual with assembly instructions

Assembly Instructions

01 793 02 99

This assembly instruction is matched to the assembly tool kit "Complete".

Individual steps are explained with tools that are not included in the assembly tool kit

"Productivity" or "Quality-relevant".

Suitable tools have to be used in this case.