Martin Langner, Introduction to Digital Image and Artefact Science (Summer Semester 2021) II. Digitisation and Data Management: Lesson 4. Digitisation in 3D (https://youtu.be/oK3y5I-oHq8) [1] Introduction [2] Object Digitisation: Methods and Perspectives [9] Content of this lecture lesson [10] 1. Visualisation Methods in 2D [11] a) 360° Photo Series [16] b) Stereovision [18] c) RTI [30] 2. acquisition in 3D [31] a) 3D Construction (CAD) [32] b) 3D Laser Scan [35] c) White Light Scan [38] d) Geometry [46] e) Photogrammetry [49] f) Handling and Tools [54] g) Good Practice Examples [61] h) Computed Tomography [64] i) Documentation Methods [68] 3. Challenges [69] a) Post-processing [74] b) Tools [76] c) Publication [85] d) Annotation [88] e) Automation [92] Conclusion [92] Current Research Questions [93] What you know and what you should be able to do [96] Literature [1] Welcome to the fourth lesson in our lecture "Introduction to Digital Image and Artefact Science". Today is literally all about digitisation in 3D. [2] The traditional method of recording collection objects, photography, offered a number of pitfalls. In archaeological circles, people liked to say: "I can photograph everything for you in the way you need it for your thesis". Uniformly illuminating a portrait creates a completely different effect than lighting it from the side.. In addition, a slightly lower view leads to a heroisation of the person, in this case the roman emperor Augustus, just as the view from above romanticises the portrayal; in other words, the importance of the angle of view and illumination is so relevant for the effect that the documentary value of photographs depends very much on it. 3] For this reason, certain standards now apply to the photographic documentation of collection objects. Above all, orthophotos of the frontal view, both profiles and the rear view are required. By orthophoto is meant a distortion-free or geometrically rectified and measurable image. Orthophotography as a standardised method of photography produces these images by aligning the camera horizontally and vertically straight to the object and parallel to its surface. The photographs are taken under uniformly soft light. The use of a telephoto lens with a long exposure time produces a high depth of field. Even with digital photography, orthophotography retains its importance as a scientific documentation method. In scientific publications, strict frontal and profile views of the objects are still common. The aim of orthophotography is to produce an image that is as undistorted and unadulterated as possible, and which can be scientifically compared and evaluated on the basis of these characteristics. [4] But what advantages do the disproportionately more complex 3D models offer? If you ask about the significance in general, you usually only get general statements, such as that 3D is cool and currently very present, if you think of film, advertising, YouTube, 3D prints or computer games, for example. Of course, this is also due to the fact that 3D tools and storage possibilities have become much better in the meantime. Anyone can create 3D models from digital photos. There is special 3D software (such as Blender), the possibilities in the gaming sector have improved greatly, user interfaces emulate the 3rd dimension and 3D printing has also become affordable. 5] On the other hand, one also hears statements such as: "3D makes nice animations, nothing more", "The 3D modellers just take a few buildings and put them together", "3D can never comprehensively represent reality / the object" or "At university we never learned how to make 3D models". The latter is not true for our degree programme, as you can see from the example on this slide. Students created the Roman thermal complex in a Blender exercise in spring 2019. But Virtual Reality is only the topic of the 10th lesson. [6] Today it will be about collection objects. Publishing these in the third dimension offers a number of advantages over the usual photo documentation: On the one hand, 3D models are globally available, reproducible at will and not dependent on the opening hours of museums (or libraries for printed publication). On the other hand, the handling of the models is simple and takes place without contact. This makes it easy to individually change the viewer's point of view (e.g. rotating, zooming, juxtaposing). This enables completely different forms of interaction with the collection objects than would ever be possible on the original objects, which can indeed be very heavy. Accordingly, measurements (e.g. to determine the dimensional uniformity of copies) can be carried out much more easily on the model than on the original. But the changeability of the 3D models is also of great importance. Thus, historical states can be restored, reconstructions can be made and fragments can be assigned. 7] However, not all problems have been solved yet: the current discussion on methods of digitisation procedures concerns three questions in particular: How can the material quality of an artefact be comprehensively documented and standardised? Which procedure is appropriate for the object in question? And: How can the procedures be automated without sacrificing conservation and restoration care? 8] From this you can already see that artefact digitisation is understood as a component of intensive acquisition of materiality. It is therefore a matter of determining the prerequisites for the historical perception of the objects in the sense of a microhistory, in order to gain further indications with the digitisation that can help with the object biography of the things. Equally important, however, is the reflection on the associated schematisation and its scientific use. Here we need to work out good practice examples for metadata and classification systems. We will discuss all these questions in the following semesters and we‘re looking for possible solutions. 9] Today we need to lay the foundations for a practical and theoretical discussion of object digitisation. So we will mainly talk about acquisition methods, explain mathematical and technical basics. What can be done with the 3D models will be discussed in the 7th lesson. Nevertheless, we will already get to know some good practice examples today. The lecture lesson is again divided into three parts. First, we will deal with visualisation methods in 2D, such as the 360° photo series, RTI and stereovision. This is followed by the main part, which deals with acquisition in 3D. Here we will also take a closer look at the geometry of 3D models. The third part, which I have called Challenges, is about the post-processing of the 3D scans, some helpful tools and the question of what a publication in 3D might actually look like. 10] Let's start with visualisation methods in 2D, i.e. methods that reflect the 3rd dimension without using real 3D models in the narrower sense. [11] You may be most familiar with 360° panoramas. With the help of the stitching technique, all-round shots with a certain overlap are put together in such a way that the eye thinks it is moving spatially. The most common type of projection is the cylindrical projection, where you get the impression of standing in the middle of a cylinder looking at its surface from the inside. 12] For the entire room, 360° x 180° panoramas in spherical or cube projection are used to depict the entire room. You know this from google Street View. But Google also uses it in its Arts and Culture project for interiors. Here, viewers built into the website are used with the help of Flash or HTML5 players. Arrows can be used to navigate to the next 360° panorama. The viewer can thus move interactively in the panorama, but only see a section in the area projection he is used to. [13] In product photography, the reverse procedure is used, so that it is not the viewer but the object that rotates. Here, too, we are not dealing with a 3D model, but with photographs that are arranged as a sequence in a kind of film in such a way that they create the illusion of three-dimensionality. [14] Here, too, 360° x 180° or x 90° acquisition is common and corresponding lighting boxes exist on the market to standardise the processes and achieve consistent quality. This is because if the position of the object shifts slightly during the shot or too few photos are taken, the jerky effect will cancel out the illusion. [15] Turntables controlled automatically by a computer or mobile phone are also on the market and can be used to achieve quite good results without much effort. The method is suitable for all types of objects, but offers limited viewing, editing and comparison possibilities. Especially in virtual spaces, these visualisations cannot be used because the light falling on them cannot be remodelled. [16] Because the eyes of humans and many animals are located at different lateral positions on the head, binocular vision results in two slightly different images projected onto the retinas of the eyes. This disparity is processed in the visual cortex of the brain to create a perception of depth. Stereovision uses this effect. Two cameras take photos of the same scene, but separated by a small distance. An algorithm compares the images for matches as it superimposes the two photos. The disparity at which objects in the image best match is used by the computer programme to calculate their distance. Computer stereovision using many cameras under fixed lighting is called structure from mo8on and is the reverse of photogrammetry. [17] In May 2018, Facebook presented an algorithm for creating 3D panoramas from a sequence of linked and aligned photos. Mobile phone cameras with two lenses can easily capture such sequences. The algorithm processes about one input image per second, resulting in an end-to-end runtime of about one minute for medium-sized panoramas. The finished 3D panoramas are very detailed and can also be viewed with data glasses and VR headsets. [18] While these techniques are more established in the consumer sector, Reflectance Transformation Imaging (RTI) is becoming increasingly important in the cultural heritage sector. This is a two-dimensional visualisation technique in which a series of photographs are created under changing lighting conditions. Originating from work on textures that depend on light sources (the 'Polynomial Texture Mapping'), RTI refers to, firstly, the method of taking photographs from the same vantage point with different directions or positions of light sources, and secondly, the acquisition of this data in the form of Polynomial Texture Mapping and the corresponding visualisation tools. These techniques were popularised by the Cultural Heritage Imaging Association. With the help of the RTI method, the surface structure of an object can be reproduced in great detail in the image. The method is particularly suitable where small or minute level differences on the surface are to be made visible, such as writing on papyri, carvings, inscriptions or flat reliefs. 19] The camera's position is not changed, it remains in a fixed position. Because of the changing light, i.e. by firing the flash from a different position each time, the way the object reflects light is changed, modelling its surface. This is done in a completely dark room so that daylight does not trigger unwanted reflections. We took the photos here of the set-up for demonstration purposes only. In addition to conventional photography equipment, all you need are two reflecting spheres to capture the position of the light source in the photo. Open source tools for processing the captured images and viewers for publishing the RTI files on the web are freely available. [20] RTI is particularly effective in the acquisition of surface features such as tool and use marks, carvings and shallow relief. The Centre for the Study of Ancient Documents at Oxford University has thus made legible hundreds of the so-called Vindolanda tablets, Latin documents on wax tablets of which only the residue of the writing on the wood of the substrate remains. [21] Similarly, the marble parium in the Ashmolean Museum in Oxford has been re-read, Ptolemaic inscriptions recorded and a new reading of the Philae obelisk in Kingston Lacy made possible. [22] The imaging process can also be perfected for smaller objects using an illumination dome. Here, computer-controlled alternating LEDs are switched on for illumination. [23] For example, a dome with 24 LEDs was used to record paintings in the National Gallery and create Polynomial Texture Maps that provided information on the conservation of the paintings. Joint studies by the National Gallery and the Tate concluded that RTI is an effective means of documenting changes in the condition of paintings and should therefore be used to assess paintings during conservation work and before and after loan. [24] This is because this non-invasive, high-resolution and powerful method is able to detect geometric changes such as holes, bends and cracks as well as colour changes such as abrasion of the canvas that are smaller than half a millimetre. [25] RTI also lends itself to the visualisation of cuneiform tablets and scroll seals, ... [26] ... as well as for the acquisition and characterisation of coin surfaces ... [27] ... or the improvement of the perceptibility of rock paintings and petroglyphs. [28] The preliminary drawings of vase painters and the relief and contour lines on red-figure vases can also be documented with RTI. Paula Artal-Isbrand and Philip Klausmeyer thus found two different types of relief lines and were able to confirm that the relief lines were laid with a brush consisting of only a few hairs. [29] ... and this is what the Dome looks like when google builds it, who use no less than one hundred cameras and 330 LEDs to do it! From this huge amount of photos, one can completely calculate the geometry of the person ... [30] , which brings us to the second part of the lesson, the acquisition methods in the third dimension. Acquisition is the measurement of 2D or 3D data by scanning, photographing, photogrammetry, tomography or any other surveying technique. [31] This is to be distinguished from 3D modelling, which always implies a complete construction drawing ex novo on the screen, which is also called Computer Aided Design (CAD). Using computer programmes, two-dimensional drawings or virtual, three-dimensional models of the objects are created or supplemented. Drawings and photographs can serve as the basis of CAD. Technologies have been developed for the film market with which complex 3D models can be created relatively easily. The file size of a 3D model created with CAD is usually considerably smaller than with other 3D methods. Unlike archaeological hand drawings, CAD is usually concerned with the ideal shape of the object and not with the variation of the smallest details. The three-dimensional model can contain not only geometric properties of the body but also volume and material properties of the object. A model created with CAD can be easily integrated into web databases or virtual environments (such as computer games or virtual museums) because of its small file size. [32] In contrast, 3D scanning attempts to create a 3D model that is as similar as possible to the object by measuring it. 3D scanners use optical devices (e.g. a laser and a camera) to measure the curvature of the surface and thus map the geometry of physical objects as point clouds or meshes. The problems associated with this should not be concealed: Since it is an optical method, it is not possible to measure what is not visible (such as undercuts). Reflective and opaque (i.e. translucent) surfaces therefore also cause problems. [33] Laser surveying 3D scanners with a short range of less than one metre focal distance are often used in industry. [34] The measurement is carried out using the intercept theorems: The distance between the laser output and the sensor (c) is known. The distance to the object (a) is measured by the sensor. This allows the exact position of the measuring point in relation to the laser source to be calculated and entered into a coordinate system. In this way, countless individual points are measured in their exact spatial position. [35] In white light scanning, the surface is not measured with a laser line, but with structured light. In this process, a kind of beamer projects a structured, spatial fringe pattern onto the object. [36] The geometry of the body can be calculated from the curvature of the lines using triangulation. The better the lenses used, the more accurate the measurement. The 3D models created in this way are therefore usually very accurate and have a high resolution. However, translucent or highly reflective surfaces are very difficult to acquire with this scanner, as are parts that are obscured by undercuts, for example. In the high-resolution white-light scan of a clay pot, every unevenness (such as chipping or paint application) is visible in the relief. The number of triangles in the 3D model is correspondingly high. [37] The white light scanner has now also arrived on the consumer market, where portable devices are also available for relatively little money. The optics in these devices is the crucial problem, which cannot meet scientific demands at the price. 38] But how exactly is the 3D model constructed? The scanner only generates a point cloud, i.e. a disorganised spatial structure of points, or better a simple list of 3D vectors between which there is no connecting structure. Additional attributes can be acquired for the points, such as geometric normals, colour values or measurement accuracy. [39] Again, with the help of triangulation, i.e. the generation of points and triangles to describe a geometric body, point clouds can be transformed into 3D models. One speaks of a 3D model when one means the mathematical representation of the surface of an object in three dimensions. It therefore consists of a collection of points in 3D space connected by various geometric entities such as triangles, lines, curved surfaces and so on. Triangulation is an old mathematical method, already known to Thales, by which a curved surface is broken down into triangles that can be easily calculated. In this process, a given surface or volume of space is approximated by a set of smaller, usually very simple elements. The resulting mesh is a simplified description of the surface or body. Due to the lower complexity, it is no longer so memory-intensive and can thus be used for further calculations, for example. Instead of a point cloud, triangulated surfaces can be displayed much faster (namely about 300,000 points per second) even with a high accuracy rate (of about 0.05 mm). Please keep in mind, however, that there can be several ways of meshing from a given set of points. 40] The basic element of a mesh is the vertex. In geometry, a vertex is the corner of a polygon. In 3D computer graphics, a vertex is a corner or node point, i.e. a point where several directions meet. Two vertices together make the end points of a line or edge, three vertices define a triangular area, etc. A vertex corresponds to a position in space in three dimensions (i.e. with three values or coordinates for positioning, often referred to as X, Y and Z) and is expressed as a vector. A vertex can usually contain other information besides the positional information in the form of a 3D vector, such as colour, transparency, or a second positional information used for texture coordinates, for example. [41] In computer graphics, vertices connected to each other with edges form a polygon mesh, grid or mesh, which has the shape of a polyhedron. A mesh is therefore a list of 3D polygons that represents a set of connected vertices. Each vertex is therefore also a vertex. The connections between the points, the edges, structure the set of 3D points. As a result of triangulation, the triangle mesh is particularly common here. 42] In 3D, the term colour refers mainly to the value that can be displayed on a screen or acquired with a camera. Often this is a red-green-blue triple of values. Each 3D point or vertex can be assigned to a colour. The colour per vertex represents either a directly displayable colour or a reflectance property, mainly an albedo. In 3D computer graphics, albedo refers to the measure of diffuse scattering power of different materials and is used for simulations of volume scattering. Its value lies between 0 and 1. [43] In addition to the mesh, a 3D model can also store the colour information separately as a texture. You have to think of this texture as a shell that dresses up the model. The exact localisation is stored as a second coordinate value in the vertices. 44] In texture mapping, a 3D surface is covered with textures. Stored as an image file, the texture thus defines a 2D image that is used to clad a part or an entire object. The pixels of a 2D texture are called texels. If you look closely, the texture file consists of arrays of contiguous texels that look like cut-up photo chips. [45] In geometry, a normal vector, is a vector that is orthogonal (i.e. perpendicular) to a straight line, curve, plane or curved surface. A straight line with this vector as its direction vector is called a normal. In a closed volume, the normal points outwards. The normal vector of a plane E in three-dimensional space is therefore a vector that is perpendicular to this plane. The normal vector of a curved surface (at a certain point) corresponds to the normal vector of the tangential plane at this point. In computer graphics, normal vectors are used, among other things, to calculate the incidence of light and reflections on the curved surface. In summary, a 3D model with texture consists of a mesh (e.g. in .obj format), a texture (e.g. .jpg) and the material (e.g. .mtl). The geometric properties of an object are stored in it as a polygon mesh, i.e. vertices, faces, normals, smoothings and texture coordinates. Optical material properties (e.g. reflection, transparency, specularity, etc.) are defined in a separate material file (.mtl, material template library), which can also contain information on texturing. 46] Instead of scanners, commercially available digital cameras can also be used for 3D acquisition. A set of 3D points (vertices) is calculated from a series of photos or digital images from different perspectives. This set of 3D points is defined in a single reference frame and in a scale or size to be determined. Corresponding computer programmes therefore search for matching points between the photographs and use them to calculate the distance and position to the camera in order to be able to arrange the points exactly in space and generate a 3D model from them. To do this, the photos should always be taken under the same conditions (such as lighting, shutter speed, white balance). Also, the object must be photographed from different camera positions and a sufficient number of overlapping photos are needed. Photogrammetry can be used to create relatively accurate 3D models, but their geometry usually falls short of a good photographic reproduction of the surface. Colour information can be assigned to each vertex by extracting colour information from the original images. This method is particularly well suited for objects whose colour and surface design are important, since a texture for the 3D model can also be created from the photos. [47] Software with so-called multi-view stereo matching algorithms automate the matching of the reference points if enough closely overlapping photos are available. [48] A textured mesh is then created from these photo series. In this way, you could also create 3D models of friends and relatives, for example. [49] On our campaigns in Vienna we placed the objects on turntables. For the heavy busts, we even made an extra turntable for Euro-pallets that can carry up to 400 kg. With three cameras that we triggered simultaneously, we were able to systematically acquire the object from all positions. In this way, the photos, each taken 3° apart, were taken in about 20 minutes. 50] Most of the time, however, is needed for the handling of the objects. The sculptures could only be moved under the supervision of a restorer. Some monuments first had to be erected with a crane. With others, unpacking the fragile and well-secured pieces proved difficult. And in the museum, we had to shine powerful spotlights against the exhibition's spotlights to achieve a reasonably usable illumination. [51] For quick, but not quite as high-quality documentation, we use an additional infrared scanner that captures the geometry, which also makes it easy to locate photos taken with the iPad. This handy scanner is also suitable for capturing completely immobile pieces. Fortunately, my co-worker was an experienced climber and was able to climb high up on the shelf without suffering from vertigo. 52] What possibilities are there for you to create 3D models photogrammetrically? There are now a few tools on the market for this. As a free service, KU Leuven has been offering Arc3D Upload for many years, where you can upload your photos and then, after a few hours, download the finished processed model via email link. If you prefer to control the process yourself, I recommend Agisoft Metashape, which is also installed on our institute computers. The programme produces photogrammetrically triangulated 3D models from your photo series and also supports the use of a turntable and projects with multiple cameras. It produces dense point clouds and complex model processing for accurate results. This makes it suitable for collection objects as well as buildings, interiors and archaeological sites. Its particular strength, however, lies in the creation of almost photorealistic textures. 53] So much for the basics. If you are interested in the algorithms behind the photogrammetric methods, please refer to the article by Michael Zollhöfer and his colleagues, who summarise and comparatively analyse the currently most common methods of static and dynamic 3D visualisation of objects and scenes and also address problems of colour rendering and light reflection. [54] Digitising sculpture at a level that provides sufficient precision to serve as a primary resource for scholarly work began at the end of the 20th century with the Michelangelo Project. A team of 30 staff and students from Stanford University and the University of Washington spent the academic year 1998/99 in Italy with the goal of creating an archive of as many of Michelangelo's statues as could be acquired in a year with a laser scanner, and to make the archive as detailed as scanning and computer technology allowed at the time. As a result, high-resolution scans of ten marble sculptures were produced in 2000-2003, including the David with a size of 2 million polygons. [55] Among other things, the project also aimed at digitally increasing the visibility of the chisel marks. On the left above you can see a detail photo without depth information of St. Matthew. The rendered images of the mesh show the same section with different illuminations and at the bottom right with per-vertex colour. [56] Laser scans of cultural artefacts were initially used for restoration issues. In 2002/03, the Visual Computing Lab in Pisa was thus able not only to measure the height of Michelangelo's David, its surface and volume, but also to analyse how exposure to falling pollutants such as rain, fog or dust had affected the surface of the statue. In addition, the centre of gravity of the projection on the base was also visualised in order to understand how the weight of the entire statue is distributed and which are the corresponding weak parts that need to be restored. [57] In this process, the 3D model was also used to map the detailed investigations. Similarly, a high-resolution 3D model of the "Pietà" was already used in 2002 to examine the damage to the statue and to guide the subsequent repair. 58] You may be familiar with body scanners from the airport. They can also be used for the acquisition of works of art, such as the bronze horse here, to record large-scale sculpture. [59] High-resolution 3D scans are often used to analyse details that are difficult for the human eye to see or systematically evaluate. The Heidelberg GigaMesh project, for example, increases the readability of 3D-scanned cuneiform tablets and uses this signal to transform the indentations into legible characters. [60] 3D-scanned models are also suitable for improving archaeological documentation, which can be translated into different types of representations by applying image processing algorithms, such as screenshots of the 3D recordings in artificial grazing light or 2D line drawings, which significantly improves the perceptibility of important details. This is equivalent to traditional archaeological hand-drawing, but more time-consuming and costly. But more on this later. [61] First, I would like to introduce another recording method. Computed tomography (CT) is a medical imaging technique that uses a computer to generate digital cross-sectional images of physical objects based on X-ray signals. This also makes it possible to get an image of the inside of the object. A large number of X-ray images from different directions and in different positions are necessary. In image and object science, digital CT sections are useful when it is not possible to see the inside of an object from the outside. This may be the case, for example, with closed vessels. The digital section can also be used for matching with analogue profile drawings. [62] Steven D. Laylock and his colleagues used X-ray micro-CT technology to scan one of the famous 19th century Cantonese ivory chess pieces with a voxel resolution of less than 9 μm and then 3D printed the model magnified tenfold; this allowed the scientists to study both the overall structure of the chess piece and the fine surface details that are imperceptible in the original. The intricate internal structure of the sphere could also be virtually dissected, printed separately and examined, making it clear that the whole was carved from a single piece. 63] If one compares CT and white light scanning, as Stefan Karl has done here, for example, the respective advantages of the two methods become clear. The interior of this aryballo can only be captured by X-ray computed tomography, while the shape in all its details, including the incised lines on the outside, is best documented by a high-resolution white-light scanner. However, these incisions could also be reproduced very well with RTI. 64] Mona Hess and her colleagues have published a worthwhile comparison of the recording methods, which they carried out on two Roman silver coins. And you can also download the booklet for our small exhibition "Analogue and digital acquisition of collection objects in comparison". 65] A word about automated documentation methods: early on, people were looking for ways to automate the creation of archaeological profile drawings of found pottery. One approach is to make white-light scans of the sherds, here six at a time on a turntable, and then extract a directional version from the projection of the outlines of the 3D scan, here of a shell fragment from Tel Dor in Israel. [66] This method is necessary because cuts through the models revealed that, in contrast to modern ceramics, there is a considerable deviation in the orientation and thickness of the walls of ancient vessels. You see here on the upper left ten superimposed sections through the vessels, from a V-shaped bowl from Bir-Safadi and below from a bowl made on a modern, fast-spinning potter's wheel; on the right there are the corresponding automatic profile drawings created from the projection. The basic problem of documentation is whether and how to document these variations. In order to do this, one must be aware of the goal that archaeology is pursuing with the profile drawings. Ceramic research is not at all concerned with documenting the actual state of the vessel, but rather wants to use the drawings to record the design of the vessel in order to be able to compare it with other vessels in terms of typology, chronology and places of production. 67] An experienced ceramic researcher reads the profile drawings like a musical notation. From small details in the curve he can immediately see what the specific characteristics of the vessel are in comparison to others of the same period and production site. That is why the foot and mouth profiles are so important. Because this is where the hand of the potter reveals itself, who unconsciously always makes the same hand movements. What happens before and during the firing, whether the vessel warps a little, for example, is not important. That is why profile drawings are always slightly straightened so as not to overload them with irrelevant factors. And on the other hand, significant details are sometimes emphasised, with a slightly thicker line, for example, or a sharpening that is overly clear at the point. Unfortunately, from an archaeological perspective, one can do little with the automatically generated profile drawings or with the results of computer tomography. Unless one is interested in the accuracy of ancient pottery wheels. Otherwise, however, important information is lost in this way. In the case of the V-shaped cup from Bir-Safadi, for example, it is very relevant whether it has a foot that is at the bottom set off from the ground or not. For archaeological drawing, therefore, one needs experience and, above all, specialist knowledge that is incorporated into the drawing. However, these semantic components cannot be distinguished without knowledge of the overall context. The computer would first have to be trained to do this. For this reason, it is possible to automatically create sections through 3D models, which would certainly be very helpful in many cases, but if the interpretation of a finding depends on a shard, I would still prefer to draw it myself by hand. If you consider the effort involved in scanning and processing the data, the time saved is probably not even important. 68] This actually brings us to very significant challenges. I would like to talk about these fundamental issues of processing and publishing 3D models in the third part. 69] A first challenge is the post-processing of the 3D scans. Typically, the available models contain too many polygons to be used in an internet viewer or virtual reality application. Reducing the number of polygons is called decimation. The validation of the simplified model must rely on scientific advice to ensure the quality of the final model. We have done a number of experiments that showed that if you have a good texture, the appearance of the model hardly changes when you decimate the surfaces. Only at a very small value of about 30,000 faces did the surface become noticeably more angular. 70] In addition to basic steps such as downsizing, smoothing (or merging) the surface also plays an important role in eliminating just such edgy effects and removing the effect that image noise produces in the photos, namely a slightly pockmarked surface. In a way, however, merging means a loss of data by calculating an average value. Therefore, one has to proceed very carefully here. Some colleagues, when there is noise in the data, prefer to sample the point clouds before generating a mesh rather than merging after the fact. [71] Such steps of scientific intervention in a 3D model, starting from the initial structured model and all resulting processed models, are called 3D processing. This includes filling holes or post-processing the texture. Any processing performed to change from one state to another must be entered into the paradata. [72] 3D data can also be enriched with information. This enrichment is an interpretation of the acquired data and is often necessary for understanding the object of study. This information can be added by colouring a part of the model or applying a texture, adding text or filters or classifying a mesh or a subset of it. Textual semantic enrichment can use thesauri or even ontologies to structure the relationships between the terms used. [73] In a sense, semantic enrichment also includes georeferencing, which is the positioning and geographic orientation of 3D models without modifying the original source data. This opera8on consists of changing from a relative position of the 3D data to an absolute position in a standardised geographical coordinate system, e.g. in order to be able to insert components into a building model with an exact fit. 74] In terms of tools for 3D processing, we use Meshmixer and especially MeshLab in the institute and in teaching, for which we regularly offer exercises and have also created tutorials that are specifically aimed at students of the humanities. [75] MeshLab and the Arc3D web service are part of a series of EU-funded international research projects on the 3D acquisition of cultural heritage; and 15 years after the start, the work done there continues to be very enlightening. [76] Lastly, let's move on to the publication of 3D models. Adobe's public domain format PDF also supports the storage of 3D models, making them relatively easy to distribute. One example is the European cultural heritage portal Europeana. [77] There you can find, for example, the Forum of Pompeii as a 3D model. [78] A large number of viewers are available for objects. In the World Wide Web, the Google standard jsc3d has established itself, which is coded entirely in Javascript and requires an HTML interface to perform rendering and interaction. Such viewers are very useful because the previews often show a photo of the original rather than the model. [79] One of the first viewers brought the Egyptian Antiquities of Macquarie University in Sydney online ... [80] And we also use the Google standard for our 3D repository, where you can view and download the scans after casts of the Göttingen plaster cast collection. [81] In the meantime, there are a multitude of 3D repositories, starting with the collections of test models for 3D development ... [82] ... to popular collections that are growing rapidly every day since it has become possible to create photogrammetric models with simple mobile phone apps. 83] One such platform is Sketchfab, which can perhaps be compared to Instagram or Facebook in terms of user-friendliness. Here, anyone can present their 3D models and quite a few can also be downloaded. As one of the few platforms, SketchFab offers the possibility of displaying metadata for the model and also provides an annotation tool for 3D models, although this is hardly used. However, the paradata necessary for assessing the quality are not requested during the upload. In our case, Geoffrey Marchal at least stated that he had created the model with ReMake and ReCap from Autodesk. Unfortunately, important information is not given. Among other things, the camera data, the number of original photos and a description of the lighting conditions are missing. But of course SketchFab, although it aggressively advertises for museums, does not want to put a scientifically relevant data collection on the net, but rather to achieve a certain broad effect with the beautiful museum objects. 84] Another reason is that, unlike digital editions, a scientific standard for 3D models is only slowly emerging. In addition to the EU projects mentioned above, the workshop "Computational Geometry and Ontologies for Cultural Heritage 3D Digital Libraries: What are the future alternatives for Europeana?", which resulted in the series "3D Research Challenges in Cultural Heritage", also played a significant role in this development. 85] In it, for example, a first annotation tool for 3D models is presented. [86] Photogrammetry also works as a crowdsourcing project. For example, the images of various photographers on the internet can be used to model famous buildings photogrammetrically. The prerequisite is that you get enough photos from different positions. [87] Then automatic 3D modelling is quite possible, as is shown here in the example of the Sistine Chapel, where methods of automatic text analysis and Google Image Search are combined to create a photogrammetric 3D reconstruction in order to combine web text with 3D shape data. Experiments at several sites show that the system is relatively successful for famous places. [88] The problem of automatic annotation of 3D models is accompanied by another difficulty, namely automatic segmentation. The so-called 3D puzzle, i.e. the partial decomposition of the models, is still a largely unsolved problem, even though there are already initial approaches. It is needed, for example, to display only the heads of statues in a repository. For this, not only must it be automatically recognised that the statue is a human figure, but also which area belongs to the head and which does not. However, the segmentation is necessary to train a neural network to automatically assign labels to these parts. [89] Another challenge is to have a significant amount of collection objects digitised in 3D in the first place. During its scanning campaign at the Victoria and Albert Museum, the Fraunhofer Institute documented the time required for the acquisition of the collection objects. It varies depending on the size and material of the objects, but at the time it was an average of ten hours per object. It also became clear that significant post-processing time was required for materials that are difficult to acquire using optical methods. Mass digitisation of cultural property thus seems to be an unreachable goal. 90] Even the Fraunhofer Institute's scanning line could not solve this problem. It is worthwhile for objects that are not sensitive to restoration and that can be easily handled. However, I know of hardly any collections where this is the case. Most museums would turn down the Fraunhofer because of conservation concerns. 91] The situation is similar with the Google Art Project, where in 2015 the highlights of the museums were scanned with a laser scanner in combination with photogrammetry. My colleagues in Vienna were so uncomfortable that they favoured our time-consuming procedure, which is currently often the only feasible way to digitise cultural assets. For the problem lies not only in the technology, but above all in the legitimate procedures in the museum. 92] Research is currently being conducted in computer science to combine a wide variety of 2D and 3D data in the photogrammetric creation of 3D models. The central challenge lies in the integration of heterogeneous information into a uniform frame of reference. It is therefore a matter of jointly analysing and processing multimodal visual data, such as photos, 3D scans and videos, even when setting texture data. Here, from an object science perspective, there is a focus on the comprehensive and object-specific acquisition and indexing of material culture. Thus, in the case of pictorial works, the image carriers should be just as much a part of the acquisition as the image, because the materiality of the objects and the functional and contextual connotations must also be taken into account. Newer data-based methods attempt to identify higher-level concepts such as shape categories or semantic attributes in geometric 2D and 3D data, and also to achieve automatic annotation of sub-areas. Again, a corpus of relevant training data is a key component in achieving this goal. One is a problem in computer science: the automatic decomposition of models into parts and their correct naming. The other is a humanities one: which parts (or combinations of details) of a representation are the carriers of meaning in the first place? How are the differences in the pictorial works to be weighted? And how can this be formalised? Accordingly, it must be further examined how much information can be extracted from the geometry of the objects and how much must be annotated. In this context, it is also important in which way the semantic enrichment of 3D models can best be implemented. The structure of scientific repositories has also not yet been clarified in the object area. Unlike digital editions and virtual spaces, we are still in the early stages of documenting 3D object data. 93] Today you have been given a kind of fast-track overview of 3D digitisation. You have learned about the different methods of spatial visualisation of monuments and collection objects. We also talked about the advantages and disadvantages of the different methods of 3D acquisition. You would still need to find out about the file formats for storage when working with 3D models. I mentioned .obj as an example, but .stl or .ply are also widely used, for example. I would also like to invite you to take a closer look at some online repositories for 3D models and compare their possibilities. For this I can recommend the first MeshLab tutorial. In practical terms, you should familiarise yourself with the workflows involved in the photogrammetric documentation of collection objects. We regularly offer exercises for this. And in order to understand what you are actually doing during acquisition and processing, the basics of geometry, briefly presented today, are also essential. [94] All this will become much clearer to you once you put it into practice. 3D acquisition from photo series (read: photogrammetry) is a matter of practice to some extent. You can also learn how to edit 3D models (like cropping, manipulating, texturing) with the help of a 3D editing programme like MeshLab. Our MeshLab tutorials will help you here. And so that the whole thing doesn't stop at the level of unscientific gimmickry, you should also be familiar with the scientific documentation of meta and paradata. [95] What is the common ground between laser scanning and white light scanning? What are the differences? Which methods would you use for the acquisition of Greek vases and why? What are the advantages of digitally acquiring and publishing collection objects as 3D models? Describe the structure (geometry) of a 3D model. Which objects can only be incompletely digitised with a laser or white light scanner? What challenges do you see for the digital indexing of artworks in 3D? [96] With a look at the research literature, I bid you farewell and hope that you were able to absorb the extensive information reasonably well.