Martin Langner, Introduction to Digital Image and Artefact Science (Summer Semester 2021) III. Analysis: Lesson 7. Shape Analysis of three-dimensional Objects (https://youtu.be/yqee_pPT_B0) [1] Introduction [4] Archaeological comparison of shapes (on Greek vases) [11] Content of this lecture lesson [13] 1. Shape Description (3D representations of an object) [15] a) Raw data [17] b) Curves and surfaces [18] c) Volumes [21] d) Graphs and feature-based descriptors [25] 2. Shape Analysis [26] a) Shape Correspondence [29] Measuring 3D models [31] Point-to-point correspondence [35] Shape Comparison [36-52] Excursus: Biometric, Informatic and Archaeological Methods of Portrait Identification [60] Artist attribution / shape comparison [66] b) Shape Classification [66] Shape, Style and Type [72] The Göttingen Project „Schemata“ [77] c) Shape Retrieval Systems [77] 3D Shape Retrieval [84] Content-based Shape Retrieval [87] 3. Modelling [88] a) 3D Reconstruction and Augmentation [105] b) Simulation of style development and cultural practice [114] Conclusion [114] Current research questions [115] What you should know and what you should be able to do [118] Literature [1] In the 4th and 5th lessons we discussed the acquisition of collection objects in 3D and their analysis in databases. Thanks to diverse and improved acquisition methods, the number of digitised artworks and finds has increased greatly in recent years. Many of these 3D models are used for visualisation or archiving purposes, but so far there are only a few projects aimed at extracting historical knowledge from point clouds and triangle meshes. This new and active research field of object analysis, which we are also working on at our department, will be the focus of today's session. So it will be about the shape analysis of three-dimensional objects, about procedures of scientific analysis in archaeology, history, art history and restoration that relate to the geometry of the objects, while we will deal with the communication of the collection objects to the general public in the 11th lesson. [2] In the fifth lesson, we had considered whether databases should not be best structured in an event-oriented way in order to be able to comprehensively represent the materiality of collection objects. These processes and cycles in the life of an object also condition the research questions for their analysis. To start from the back: We have already discussed the presentation and publication of collection objects in the lesson on databases and we will deal with their visualisation in the 11th lesson. A reconstruction of different states and collection events can be simulated digitally. We will talk about this at the end of today's session. Deposition and excavation concern both the analysis of space and time, which we will discuss in the coming lesson, and questions of virtual reality in the 10th lesson. The 9th lesson will deal with questions of statistics and network analysis. It can be used to examine trade relations as well as questions of the regular use of objects. Today, we will start with shape analysis, which can be used to draw conclusions about the design and manufacturing process of objects. 3] Depending on the orientation of the research question, various forms of object digitisation come into question. We dealt with them in detail in Lesson 4. 4] These capture methods already offer analysis functions. For example, one can compare the hand drawing of a Greek vase with the section through a 3D model, [5] by superimposing the two drawings in an image processing programme. [6] or by unrolling the image of the vase, a 2D representation of the three-dimensional object is obtained, which can now be grasped visually - without having to turn the vessel - at a glance, so to speak, even if this admittedly does not correspond to ancient perception at all. [7] Profile drawings have also always been used to analyse workshop relationships. Here, for example, it is clear that all the vessels are so similar that they must have come from one workshop, which is indicated not only by the general sweep but also, above all, by the foot profiles. [8] Such questions of similarity are best investigated by excluding other stylistic influences, for example by comparing the foot profiles of one potter with those of other workshops of the same period. [9] But manual analyses can also be carried out in the 3rd dimension. In the case of a fragment in Graz, for example, it was not clear whether the fracture surfaces would exactly match a piece in Göttingen, which is why it was scanned and the printout sent to Göttingen. [10] And at the Exekia conference in Zurich, the suspicion arose that the amphora in Philadelphia might include a handle that is in Göttingen. The fracture surfaces are so clear in the 3D scan that a new reconstruction of the vase will now be undertaken. And we in Göttingen possess such an original from the hand of Exekias. 11] But today we would like to ask about computer-assisted analyses that are possible via the shape of the digitised object. In doing so, I can only touch on very fundamental questions of the Digital Humanities and invite you to think about them as well. For in all of this, we are interested in which research questions can actually be pursued with digitisation. And specifically: how and to what extent can form-analytical approaches be more stringently comprehended through digitisation? The transfer of visual forms into language is an error-prone translation process that does not do justice to the medium "object" in its spatial form. But to what extent can artefacts be acquired non-verbally with digital methods of pattern recognition and to what degree of accuracy? And in general: what new possibilities for knowledge arise from digitalisation? 12] This lesson again has three parts: We must begin once again with an overview of the different data representations of 3D models. The main part will deal with shape analysis, where we have to distinguish between shape identity, shape similarity and shape retrieval. Finally, modelling as an analytical category is about modifying the 3D models in such a way that one gets a new access to the objects, be it through 3D reconstruction and addition of the objects themselves or through simulation of style development and cultural practice. [13] Let us start with shape description, that is, the data-based representation of the objects, by which we mean the data representation of 3D models that can be described in terms of point clouds, curves and surfaces, in terms of their volumes or abstractly as mathematical features. [14] There are significant differences between 2D object recognition in images and 3D object recognition in a 3D repository. In 2D image analysis, the typical use of features is to describe an object independently of the way it is seen by a camera. Features found in images of the photographed object are persistent elements in the form of uneven lines (edges and corners) or photometric properties (texture). Such representations simplify the task of shape analysis by reducing it to simple image processing operations, losing information about the 3D structure of the object that cannot be clearly acquired in a 2D image. [15] In 3D shape recognition, the features measured are usually based on the geometry rather than the appearance of an object. This is because shape correspondence and similarity problems require features to be stable under natural transformations that an object may undergo, which may include not only changes in positioning but also non-rigid bending. The three-dimensional shape of an object can exist in very different representations. Among these, the point cloud is the simplest representation of three-dimensional data. Optical acquisition devices such as scanners provide point clouds as raw data. The representation of the object consists of a disorganised spatial structure ('cloud') of points in space. Information about edges, volume and surface of the objects to be represented are not contained in the point clouds. Curved edges and surfaces can only be represented with a very large number of points, which leads to a correspondingly high memory requirement. However, object surfaces can be generated, e.g. by stretching polygons between the nearest neighbouring points. Since the points in space cannot be unambiguously combined into surfaces, there is a certain scope for interpretation. [16] In combination of RGB colour camera and infrared measurement, motion or depth images can be generated, which can be stored in a depth matrix with VGA resolution, to which an RGB image can be assigned. The depth images can also be converted into meshes. 17] As mentioned, object surfaces can be generated from point clouds by spanning polygons between the nearest neighbouring points. The connections between the points, the edges, thus structure the set of vertices. The object is thus described by its surface, namely a list of 3D polygons, which represent a set of linked 3D points. The number of surfaces determines the size and accuracy of the model. 18] Volume models represent the objects as volumes. This also allows objects to be described that do not have an explicit surface, such as density fields or 3D grid arrangements. In modelling, each object is described as a compilation of elementary objects. The voxel model represents the volume of an object in numerous geometric information units of the same size (called voxels), usually as a cube. Accordingly, the accuracy of the representation depends on the number of voxels. In a way, voxels form the 3D counterpart to pixels. Voxels contain information about whether they are empty or filled and what colour the filling mass is. Material properties can also be stored there. The position of a voxel within the representation is usually determined by a hierarchical process. [19] A common way to organise the voxel data is to use an octree. An octree is a hierarchical 3D data structure in which volumes are represented as a rooted tree whose nodes have eight (or no) direct successors. It consists of many cubes of different sizes with the attributes "full" or "empty". The "full" leaves of the tree can also contain further information such as colour, transparency, etc. 20] As a kind of reverse photogrammetric acquisition, a series of 2D images from different camera positions can also be created from a 3D model and evaluated with 2D object recognition methods. Multi-view has the advantage that neural networks trained on millions of photos can be used to identify the 3D figure. [21] 3D models can also be represented directly as mathematical objects, where the geometric features of the object are described in the form of graphs. For shape-based analysis of an object, it can be useful to find such a computational representation of the shape for which an index can be built to efficiently answer similarity queries. First, there are shape distributions, which represent the shape of a 3D model as a probability distribution taken from a shape function measuring geometric properties of a 3D model. Probably the best known method for such a shape distribution, calculates the probability distribution of Euclidean distances between pairs of randomly selected points on the surface. The figure shows the so-called D2 distributions for 5 tanks (grey curves) and 6 sedans (black curves). You can clearly see how well the classes can be distinguished. [22] Such distributions can, of course, also be represented with much more differentiated calculations in three dimensions, which then reflect the shape quite clearly. The advantage of such graphs is that the shape of the resulting distributions is invariant to similarity transformations, noise, tessellation, cracks, etc. [23] Another method chooses to skeletonise the object. In general, the idea is to derive skeleton curves from a 3D object such that each curve represents a significant part of the object. These curves are then converted into an attributed graph representation (a skeleton graph) that can be used for indexing, matching, segmentation, correspondence finding, etc. A well-known example demonstrates the procedure on an aircraft, where a skeleton diagram is finally created via a voxelised representation and reduction to a centre line, the so-called skeleton, which can be represented as a sum of graphs. Admittedly, this procedure leads to more meaningful results for spatially extensive objects than, for example, for the compact Stanford Bunny. [24] Feature-based descriptors represent the shape with a single descriptor consisting of a d-dimensional vector-valued function. To do this, they extract a large number of significant geometric features, compute feature statistics from them, and convert them into a point in high-dimensional space. In this way, two shapes are similar if their descriptors are close to each other in space. This compilation makes it clear that for shape analysis we are dealing with quite different 3D representations of an object, where the spatial physicality of an object is sometimes represented as a sum of points or depth matrix, sometimes as its outer surfaces, sometimes as its volume, sometimes as a sum of 2D representations and sometimes as a mathematical structure. Depending on the question, one or the other representation may be more suitable. In the digital humanities, for example, it is common to work with triangle meshes or point clouds, in medical applications with volumes and implicit representations, while 3D object recognition tends to work with 2D representations and feature-based descriptors. [25] Knowledge of these representations is important in order to assess the analysis possibilities that can be made about the shape. But which methods of computer-aided form analysis are already established and which are still under development? How does one even have to proceed in order to teach the computer what we have already learned in infancy, namely to distinguish shapes? 26] In general, object analysis faces two basic problems: that of determining the identity of two objects and that of determining the similarity between two objects. Extensive correspondence between two or more objects is achieved by shape comparison. Correspondence underlies many pattern recognition applications where we need to distinguish between different geometric objects, such as 3D face recognition. [27] The broader Shape Similarity analyses objects on a geometric and semantic level. The term shape analysis sometimes refers only to content-based shape recognition, sometimes as a generic term to both types of shape analysis. Due to the availability of large public domain databases of 3D models, such as sketchfab or the 3dwarehouse of sketchup, the problem of shape similarity has currently moved strongly into the focus of research. This is because they have significantly increased the demand for shape search and retrieval algorithms that are able to find similar shapes in the same way that a search engine responds to text queries. [28] From this follows, thirdly, shape representation or reconstruction, i.e. the automated generation and modification of shapes, whether by texturing, by morphing or by composing them in virtual spaces. [29] A simple analysis of objects comes about through dimensional comparison. The basis of such comparisons are the measurements on the 3D models. With the help of Meshlab, it is very easy to perform measurements on a 3D model. The interactive point-to-point measurement of parts of a 3D model, such as the length of the lower leg, is easily done with the Measuring tool, a 3D ruler. The basic properties of the model, which would be preserved even under continuous deformations such as stretching, compressing, bending or distorting, are displayed with the filter "Compute Topological Measures" and so you get measurements such as the number of corners, edges and faces, unreferenced vertices, the number of holes, and so on. The geometry of the mesh can be called up with the filter "Compute Geometric Measures". Here you will find measures in relation to the bounding box, a cuboid spanned at the outermost vertices, the volume and inertia of the object as well as coordinates related to the centre of gravity of the object. But remember that many measurements are only useful on watertight models, and that if you have not converted the model appropriately to mm or cm, Euler's number is used here as a reference. [30] You can also use the "Compute Planar Section" filter to calculate a planar section through the mesh. When the resulting polyline is closed, the result is filled and also the area representing the section is saved as a mesh so that one can continue working with the section. In my example I put a plane at the height of the Y-axis through the model. Other geometric information (such as curvature, geodesic distance or local vertex density) can also be calculated using automatic filters for 3D models. [31] Correspondence problems often arise in shape synthesis applications such as morphing. Morphing refers to the transformation of one image or object into another using additional intentional distortions. Morphing algorithms try to find significant elements between two images or objects and distort them during stepwise blending so that their contours are made to match. Thus, in order to transform one shape into the other, one needs to know which point on the first shape should be transformed into a point on the second shape. [32] A related problem also exists in the photogrammetric acquisition of objects. The so-called registration determines the matching points on the photographs, but explicitly looks for the deformation that brings one shape into the other. Registration thus refers to the process of finding a spatial transformation (e.g. scaling, rotation and translation) that aligns two sets of points (3D point clouds or 2D pixel coordinates) with each other, e.g. to merge multiple data sets into a globally consistent model. [33] A fundamental problem in shape analysis is correspondence, where relationships between similar points must be found on two or more shapes, i.e. a correspondence between shapes must be established by defining a large number of matching points. Such an alignment moves different meshes into a common reference system that can be used to manage large sets of range-maps. This can be done manually, and MeshLab offers two functions for this: the point-to-point alignment and the manipulator tool. We have explained exactly how to proceed in a tutorial on our Youtube channel. 34] Subsequently, you can use the Hausdorff distance to measure the geometric deviation between two 3D models and visualise it in a false-colour representation. The distance named after Felix Hausdorff determines how far apart two subsets of a metric space are. The Hausdorff distance thus denotes the largest of all distances from a point in one set to the nearest point in the other set. [35] The method is also called tolerance-based shape comparison. For tolerance based shape comparison, meshes are usually used and each polygon of two polygon meshes is compared with each other. The target polygon is normalised to fit as closely as possible into a bounding box of predefined size. Then triangulate the zone between the outer and inner polygon and measure the distances between the polygons (you may have to use the mean value). The result can then be output as a false colour table. 36] Let me explain the advantages of the method with an example from our research practice. Once again, it is about face recognition. The fundamental question of how to recognise an individual has been a subject of interest for image science and computer science for a long time, and in the meantime facial recognition software is mostly used for this purpose. But can these methods also be used for ancient portraits? As a particularly illustrative example, I have examined the frontal views of the first five Roman emperors belonging to the Iulio-Claudian dynasty for family affiliation on the site twinsornot.net. The result is sobering and does not at all correspond to our first impression. Tiberius, the only emperor who is not related to Augustus by blood and therefore clearly borrows in his portrait conception features from the Augustus portrait, still achieves 86%. The other results are even more irritating. Especially the one hundred percent correspondence between Augustus and Claudius is not comprehensible at all. 37] The result can be explained by the procedure of biometric methods. Here, a few measuring points are collected with which invariable characteristics such as the shape of the eyes, the distance between the eyes, the length of the nose and the shape and distance to the mouth can be measured and evaluated. [38] However, our usual registration of persons is based not only on biometric criteria but also on external, variable characteristics such as clothing, hairstyle and beard, but also skin colour, fullness of the face and age characteristics. That is why you have certainly recognised Angela Merkel AND Elizabeth the Second of England on the left and Putin AND Obama on the right without any difficulty. 39] The ancient portraits, however, are much more a means of expressing certain ideals than the actual characteristics of the sitter. Augustus, as is well known, is still depicted as a youthful, and thus ageless, 70-year-old man, in order to illustrate the timelessness of his reign, which was propagated as the return of the golden age. Although his portraits are easily recognisable, especially by the characteristic curling motif of fork and tongs above the forehead, Augustus is said not to have attached any importance to carefully combed hair in public appearances. And Tiberius is reported to have had particularly large eyes, receding hair and a severe skin rash. None of this is reflected in his portraits. The discrepancy between his real appearance and the portraits must have been so great that Caligula ordered that his statue in the Forum should always be dressed in the same clothes he was wearing on the day in question. Thus, the emperor's portrait is not a realistic rendering of the respective person's appearance, but a visual reference to qualities such as experience, energy, dynastic affiliation or wealth, to name just a few long-term trends in Roman portraiture. [40] That the curl schemes could also be used as features for an automated recognition of the emperors and their portrait types was already presented by Japanese colleagues in 2013. They were able to show that pattern recognition methods on 3D scans can be used to assign the forelocks to specific classes or subgroups. However, they did not investigate the design of the face. 41] The work of Michael Pfanner is also important here. In 1989, he showed by comparing silhouettes that the individual types that can be determined by curl schemes are often in an exact replica relationship, and was thus able to reconstruct the copying processes in the ancient sculpture workshops. An important result of his investigation is that the copyists oriented themselves primarily to the profile line of the face. Apart from practical considerations, this may be due to the fact that the emperor's portrait was mainly used on coins, and there in profile view, throughout the empire. A more precise comparison of 3D scans now makes it possible to gain an even better acquisition of the production processes and to verify Michael Pfanner's theses. 42] For the following dimensional comparisons (the so-called tolerance-based shape comparison), I followed Pfanner's result and first manually aligned the two 3D scans of the portraits to be compared with the profile line of the face using the programme MeshLab. The differences can already be compared quite well in MeshLab. For example, it is possible to carry out a target/actual comparison by calculating the Hausdorff distance and to map the differences between the two models using a colour table with a tolerance band (here: maximum deviation 2 mm). [43] In doing so, the degrees of deviation are projected onto one of the two models in a fixed colour scheme. This visualisation of the deviations by means of a false colour table allows a simple interpretation of the results. Reading these computer-generated distance visualisations takes indeed some practice: But basically, green areas signal a high degree of correspondance, with deviations in the tolerance range of 0-0.5 mm (light green) or 0.5-1 mm (dark green). Areas marked in red already deviate from each other by 2-2.5 mm, while non-coloured parts are positioned even more strongly, i.e. more than 2.5 mm apart. Places where the two models intersect are also shown in light green, as there is a great proximity to each other here. These intersections are easily recognisable, as for example at the neck section, due to the fast change of colours. 44] With the help of the programme OPTOCAT, which was included with the scanner, the two models can now be metrically adjusted. For this purpose, the two meshes are aligned with each other according to best fit within a certain search range. For example, with the two Augustus heads in the type Louvre MA 1280, the result can be significantly improved within the search range 1-5 mm compared to manual alignment. With a higher search range, however, the metric adjustment programme also tooks into account areas such as the completely differently aligned busts, which again leads to a weaker distance measurement in the face. [45] How strong the differences are with this procedure for completely different heads becomes clear if we take another look at the first five Roman emperors, who all deviate very strongly from the portrait of Augustus in the Prima Porta type. Here, however, it also becomes apparent that - in contrast to what can be determined by biometric methods - the design of Tiberius' face is completely similar to that of Augustus. Only the design of the eyes deviates from that of the first emperor, while the face of Claudius is clearly different not only because of his age features. It is significant that the biometric methods make the distances of eyes, nose and mouth the basis of the similarity determinations, whereas here the surface and thus the entire face are now taken into account. Tolerance-based Shape Comparison is thus much more suitable for an analysis of emperor portraits than biometric methods. [46] And to anticipate the results of our measurements, i show you a table of shape comparisons concerning Augustus heads in the Prima Porta type and the Louvre MA 1280 type. If we now compare the Augustus portraits with the help of the Shape Comparison, we are surprised to find a very high accuracy. Deviations of up to one millimetre, which are marked in green here, may certainly occur in the artisan copying process without being intended by the sculptor. But in our examples it is the astonishing accuracy in copying that seems very remarkable to me. 47] As Michael Pfanner has already observed, one usually orients oneself to the profile line, i.e. the position and shape of the forehead, nose, mouth and chin. While in 1989 he was only able to document his results in the second dimension with silhouettes, it is now possible to prove his observation in the third dimension as well. Through our shape comparisons, it also became clear that the ears are generally in the same place, although the details vary greatly. Basically, like the curls, these seem to have been worked more freely and without using measurement points. [48] Larger deviations are actually only apparent when the head is lowered or turned to one side of the body. Here the sculptor seems to have integrated foreshortening into the portrait design. This means that he has slightly shifted the measuring points or made partial copies of the respective areas of the face. [49] With regard to faces that are almost identical in size, the heads can be grouped into subgroups for which it can be assumed that they depend on a common model, precisely because they differ more from the faces of the other subgroups. The upper row was found in Rome, while the heads of the lower row, with one exception in Chiusi, were found mainly in the eastern Mediterranean. In the future, there is a high potential for studying this on a broader basis and taking into account the excavation sites. 50] Contrary to what most researchers postulate, there is a striking dimensional accuracy in the elaboration of the face, while the curls of the hairstyle are much more freely designed. This applies not only to Augustus, but also to the portraits of other emperors, as will be demonstrated here using the example of four heads of Claudius in the main type, whereby the deviations can mostly be explained by modern additions. Michael Pfanner, himself a trained stonemason, explained the use of the copying process for the portrait with the security that the copying process offered the ancient sculptors. One wonders, however, why one then ventured into such uncertain terrain when designing the hair and worked much more freely here. 51] Precisely because of the exact copying process, it is probable that the parts that deviate from the model were deliberately changed in order to strengthen the desired expression in the face. And this explains why the eyes in particular, but also the area of the mouth, were modified more frequently, probably to change the emperor's expression. As in the case of facial recognition in the brain, the sculptors evidently distinguish between the identity of a person, for which they use the copying process precisely, and his or her effect on other people, which is produced by facial expression, eye position and hair design. 52] If our observations on the major deviations in the eye area, which have so far only been made on a relatively small basis, are correct, then this is where the sculptors were most likely to see the possibility of improving the expression of the face in their sense, and these ambitions can once again explain why biometric methods must fail in the case of intentional pictorial works. However, they also show that 3D shape analysis can lead to new results even in traditional fields of research. 53] For example, it is relatively easy to use this method to check whether two terracottas were taken from the same mould, i.e. model. This is certainly the case with these two figurines of Eros in the original archaeological collection in Göttingen. [54] Using the same method Elisabeth Trinkl has been able to prove that a number of so-called head vessels must have come from the same potter's workshop. [55] Because of the high accuracy of our scans, it is even possible to prove that two plaster casts are taken from the same mould. This is because the cast seams have to be worked off manually, and there are greater deviations in places along the cast seam. So it was possible for us to show that the Large Herculaneum Woman in the Goethe House in Weimar was made with a cast that had already been in use during Goethe's lifetime. [56] And sometimes one also comes across new research results by chance. When comparing two plaster casts of a bronze bust in Dresden, it turns out ... [57] that between the creation of the first cast in the 1950s and the newer cast, the bust must have fallen down. In any case, the 3D data analysis revealed an indentation on the right side of the head, which even experts had not noticed before. 58] The plaster casts, which reproduce the originals very accurately, play a special role here, as our comparative measurements of originals and casts from different moulds and in different materials has shown. The scans of casts can therefore be used for dimensional comparisons just as well as the scans of the originals. [59] The same conclusion has already been made by Bernhard Frischer, who examined the original and the cast of the famous Laocoon. [60] Most statues of classical Greek art were made of bronze and are lost today. However, many were already famous in antiquity and were copied relatively exactly in marble during the Roman imperial period. Shape comparison is an excellent way of extracting the Greek original, so to speak, from a series of Roman copies, because in programs such as MeshLab one can superimpose the 3D scans and compare them with exact dimensions. 61] Statues that survived in many Roman copies are likely to go back to a famous Greek original. These include, for example, the Diadoumenos, an athlete putting on a bandage, which is considered to be the work of the Greek sculptor Polycletus, and the Amazon in the Sciarra type, which is usually attributed to Kresilas, although an Amazon has also survived under the name of Polycletus. Japanese colleagues have placed the recessed left foot of the two statues into each other in a 3D scan and found a great similarity. However, whether this similarity of design can be attributed to the same Greek artist, the same Roman copyist or a general convention of representation in antiquity cannot be ascertained in this way alone. To do so, one would have to examine a large number of feet on ancient statues. [62] The same problem arises with the heads of such Roman copies. András Patay-Horváth, for example, has tried to show that the Ares Borghese, which he compared with the Doryphoros, was also created by Polycletus. For their profile line shows striking similarities. In principle, I think this is a good method, but it will only produce valid results if it is carried out on a large material base, because the eye area of the two heads is very different. 63] Already in 2007, a bronze horse attributed to Benvenuto Cellini was scanned with a similar shape comparison and the contour was compared with a hand drawing by Leonardo by rotating the 3D model back and forth on the drawing in the background until the head or the body matched the drawing relatively well. The result was an impressive similarity between the two works in outline. At the same time, it was also possible to show that the same similarity did not apply to similar Leonardo drawings or similar bronze statuettes. The probative value of such analyses is admittedly controversial. For if a work of art is by one artist rather than another, this does not say that other sculptors, such as in this case pupils and successors of Leonardo, are not possible authors. 64] In the case of paintings and drawings, individual small details are used to determine the painter, which, like a handwriting, reveal the particularity of an artist. András Patay-Horváth and Leif Christiansen have tried to apply this method to sculpture. Using the example of facial features such as eyes, nose and mouth, which were extracted from the 3D models, the statues of the Temple of Zeus in Olympia were examined to see whether the deviations in the design speak for one or more sculptors. For this purpose, the mean and standard deviation of the distances between two standardised points, such as the width of the eyes, were calculated. The authors came to the conclusion that the sculptures were very probably made by the same hand. However, this is difficult to reconstruct on the basis of eight eyes. We cannot yet say for sure how large and significant the variance in eye design can be and whether artistic design intent plays a role in deviations. Therefore, the relevance of such individual measurements is difficult to judge. 65] A comparable study was carried out by the same researchers at the Siphnierschatzhaus. The surface of the reliefs has suffered so much over the past one hundred years that the scans were made on a cast of the year 1913 in Göttingen. At the Siphnierschatzhaus it is widely agreed that the sculptures on the east and north friezes, which you see here, are by the same master, while the others are likely to have been executed by other craftsmen. Indeed, the hair design is very similar on some figures (East Nos. 1 to 5, North Nos. 31, 35 and 50), all of which have angular channels with a flat underside between the curls, while others (East Nos. 16 and 17) have rounded channels. Something similar can be observed in the folds of the garments. It was concluded that even if the same master was responsible for the overall design of the East and North Frisians, some sections were worked by different sculptors. However, the question arises as to whether different folds are not also due to motifs. Here, too, it would first have to be fundamentally investigated how great the variance can be with one sculptor, and whether and to what extent such measurements are relevant for a master attribution. [66] All comparisons of shape very much require expertise in image science. This is because differences in shape can have a variety of reasons. Archaeology and art history understand 'fashions' (or 'tastes of the time') as snapshots of a process of continuous change and therefore use them for dating. This is because they assume that the way of using, of consuming things is subject to time-related patterns that are reflected not only in behaviour and thinking, but also in the design of objects. [67] While contemporary taste is not an action but an attitude that is usually expressed subliminally to unconsciously, design is an active and reflected process of shaping, but the two influence each other, as producers also participate in contemporary taste. 68] Style refers to the formal design of things as a function of time and thus indirectly of social or cultural changes. In stylistic analysis, therefore, one explores changing forms, their development and the sum of common characteristics or elements of form. Stylistic features (i.e. individual characteristics) can express themselves on several levels. Thus there is the style of an epoch (period style), of a region (landscape style), of a genre (genre style), of a workshop (workshop style) and of a work or an artist (individual style). [69] The term "development" is used in visual and artefact studies to distinguish between earlier and later stylistic elements. In this series, a certain type of an Attic jug develops from rather flat to more pronounced shapes. The handle is increasingly curved, the mouth runs in a more uniform sweep, the shoulder is more distinct, and the body rises more from the foot. [70] While style encompasses changing forms and the sum of common characteristics or elements of shape, type denotes a fixed shape of relatively long temporal duration. A type is determined by abstracting individual forms and identifying their characteristic common features. Beazley, for example, distinguished ten types of jugs, which differ in the shape of their handles, mouths, bodies, etc. The size, however, does not play a role in determining the type. [71] Determining types does not only serve the purpose of bringing order into the material. With the determination of types and styles, the image and artefact sciences attempt to extract patterns from the sum of the remains of past societies, from which conclusions can be drawn about the conditions at the time. For example, shape analysis can be used to determine workshops and draw their export products on a map in order to better understand the trade relations of the time. In this sense, the image and artefact sciences always used forms of pattern recognition to describe artefacts and images, although they tended to call it structural analysis, typology or seriation. But it is always about classifying phenomena and placing these patterns in a larger context. [72] For me, this raises the question of whether the methods of archaeological shape analysis correspond to methods of digital pattern recognition. We want to investigate this in the project "Schemata", which Lucie Böttger and Alexander Zeckey are carrying out together in their dissertations. Using the example of Hellenistic terracotta figures, archaeological concepts for describing similarity and machine learning techniques for classification will be compared. The resulting discussion has two goals: The first is to provide archaeology with non-verbal forms of description that make it possible to classify not only typological dependency relationships, but also other degrees of similarity, and to enable a clearer overview of ancient perceptions of terracottas in terms of types, variants and motifs. The second goal is to significantly improve the object mining process so that a large percentage of the data on objects in a collection can be automatically stored in databases in the future. [73] This is implemented in the form of a data pipeline with step computing as the core of the workflow, in which various processes are developed and fine-tuned to suit the case study. [74] Archaeologically, the problem is the question of semantic similarity. The terracottas shown here belong to the same coroplastic type of a standing woman fully wrapped in her cloak, with her right arm bent towards her neck and her left resting on her hip. At the same time, the right leg is placed to the side in a relieved position. It is unclear, however, how this posture is to be interpreted and whether, in terms of content, the swing of the body or attributes such as the sun hat do not play a greater role than the posture alone. Lucie Böttger tries to find this out by analysing the grave contexts in order to be able to determine which features the informatic pattern recognition should learn in the first place. [75] Informatically, the aim is to exploit as much information as possible contained in the figures for the form analysis. For this purpose, Alexander Zeckey creates and evaluates various already known methods in the field of shape recognition procedures and extends them primarily with posture comparisons, a 3D-on-2D unwrapping and his own system with automatic extension and weighting procedures. In this way, we develop a possibility to determine and evaluate the degree of similarity. [76] This should of course also help to find the countless terracottas in a database more easily. [77] The retrieval of models from large 3D repositories is currently a major field of research. Here, too, most platforms work keyword-based, i.e. by entering search terms. [78] However, a graphical input is closer to the visual character of the objects. That is why work is being done on sketch-based shape retrieval, where the user simply sketches briefly what he wants to search for. 79] The foundations for this were already laid in Princeton in 2004. As you can see, inputs that capture the core of the object, such as sections through the table and a top view of the tabletop on the left, are particularly successful. An input that is too detailed and shaded, like the sofa on the right, contains too much information that could also apply to chests of drawers or bananas in a certain way. So the necessary abstraction we talked about earlier is also required here from the user of the search engine. [80] And to give you an amusing example of how things would look in the real world if we were not able to abstract, let us refer to the works of Tom Curtis. They clearly show that a sketch-based query would in itself be child's play. [81] This determination of shape similarity through abstraction is also the central method in Shape Content Collection Exploration. Vector graphics, for example, are composed of graphic primitives such as lines, circles and polygons. The same is possible in three-dimensional space. You may know this from CAD programmes or from blender, where geometric primitives are used to construct objects. Accordingly, 3D shapes can also be broken down again into basic objects in order to describe them. These basic objects correspond roughly to the bounding boxes of 2D pattern recognition, whereby the degree of abstraction that is necessary for this influences the degree of similarity. The larger the reference frame is chosen, the more shapes that are different in detail can be assigned to it. In the case of furniture or animals, for example, it is characteristic, however, how strongly the basic form or posture still shines through in highly abstracted boundaries. Currently, attempts are being made to have such form abstractions found unsupervised by Deep Learning algorithms. [82] A remarkable approach for searching in 3D repositories is to represent a shape as a collection of basic objects (quasi as geometric words) and to use the well-developed methods of text search such as the "bag of features" (or "bag of words"). Such approaches are widely used in image search and were also introduced into shape analysis ten years ago. The construction of a bag of features is usually done in a few steps. First, the shape is represented as a collection of local feature descriptors. Then the descriptors are represented by geometric words from a geometric vocabulary, the codebook, using vector quantification. Here, the geometric vocabulary or codebook consists of a set of representative descriptors that are precomputed in advance. In this way, each descriptor is replaced by the index of the closest geometric expression in the vocabulary. The calculation of the histogram of the frequency of occurrence of geometric expressions gives the bag of codewords. Shape similarity is calculated as the distance between the corresponding bags of features. [83] A very early example of Content Based 3D Shape Retrieval is represented by the dissertation of Anestis Koutsoudis, who developed a shape-based similarity search for Greek vases. Unfortunately, his website (http://www.ceti.gr/~akoutsou/akoutsou_phd/) has not been in operation since 2010. It offered the possibility to search for similar vases in a database of 1012 vessels by uploading a 3D model or a freehand sketch of the vase done with the mouse. [84] Koutsoudis offers his sample set of Greek vase shapes for download and also discloses the code he uses to determine the variations within a vessel shape. In addition, he has developed a tool to generate 3D models from profile drawings: http://www.ipet.gr/~akoutsou/qp/. We also have him to thank for a virtual museum for Greek vases that was already created in 2009, in which one can move the ceramics in the display case and have them analysed by computer: https://www.youtube.com/watch?v=K5t5FgzME5c. [85] Arik Itskovich and Ayellet Tal used a parallel surface matching technique to classify and query shapes; within a database on Hellenistic vascular pottery and lamps, they were thus able to search for specific figure types. [86] And with a contextual search based on learned room layouts, they try to find related objects, in this case pieces of furniture, by having the user mark the area next to another object or in the room where the searched object should be placed. It is easy to imagine how Egyptian pyramids, Roman forum complexes or Gothic churches could be reconstructed automatically in this way in the future. 87] This brings us to the third part of the lecture, which is about modelling as an analytical category. So it is now not about the analysis of the shape, but the analysis of the objects by changing the shape, about an explorative procedure with which one puts the object into a state that can be analysed. [88] As early as 2004, Thomas Funkhouser and his colleagues had presented an application with which one can "intelligently" cut out parts of 3D models and reassemble them. For example, new pieces of furniture can be designed and antique statues can be added. Depending on which part of the object you focus on, i.e. here rather the backrest or the feet of the chair, different suggestions are made. In view of the continuously growing 3D repositories and the possibilities offered by Deep Learning methods, research is currently being conducted into automating this approach and leaving the disassembly and assembly of the 3D models to the computer. [89] Corresponding software has been available for ten years and has been continuously developed. With MeshMixer, for example, you can also easily put your own hands on your 3D models. 90] In Virginia, a toga statue of Emperor Caligula was digitally reconstructed in 2010/11 by re-modelling the missing parts. At that time, the working time for such a task was still a whole year. First, the marble statue had to be digitised with a laser scanner. [91] At the same time, archaeologists searched for models of how to fill in the missing parts as correctly as possible. For the left forearm they took a statue in Copenhagen, for the right forearm they oriented themselves on the bronze statue of a Camillus in Rome, the nose was completed according to another Caligula portrait and for the colouring they relied on traces of colour that could be reconstructed on the Caligula portrait in Copenhagen and on mummy portraits from the Imperial period. [92] The ear was constructed in several steps. [93] The toga folds were also added, [94] which, especially in the case of the umbo, contributed to a much better overall effect. [95] The left forearm was first modelled as a whole before the toga was placed over it. [96] In the same way the right forearm was modelled before the tunica was added. [97] And the sinus, i.e. the curved hem of the cloak, was also added to complete the figure. [98] The overall result, after colouring, was a very respectable model of the original state, which can at least refer to antique models with regard to the additions. 99] To give another example. 3D scans also help in the restoration of pieces that have been damaged by the effects of war, as here with a tombstone from Palmyra. Here, too, it was possible to reconstruct the lost part and make a 3D printout. [100] Sometimes, however, parts have to be removed the other way round. Jack Wasserman used a high-resolution 3D model of the Florentine Pietà to analyse Michelangelo's motivation for destroying the statue. He was able to virtually remove the extremities that were added later and thus determine that the result is similar in form to Michelangelo's later works. Therefore, he concluded that Michelangelo's intention in chopping off parts was to create a different sculpture rather than to destroy eight years' work in one night out of anger. Such simulations will not end centuries of art historical debate, but they do introduce a new piece of evidence into the discussion. [101] A larger-scale project to reconstruct the Circus Maximus also involved the re-erection of an arch dedicated to the Emperor Titus. For this purpose, the components were graphically documented with the help of photogrammetrically acquired, textured 3D models. On the basis of a local geometric analysis, a proposal for the archaeological and architectural reconstruction and anastylosis of the fragments could be worked out: i.e. not a try and error reconstruction on the screen, but a positioning of the components on the arch calculated on the basis of the geometry of each fragment. 102] The Duke University project for the reconstruction of Trajan's Forum is even more ambitious. Here, each component is scanned and its original place on the building complex is determined like a puzzle. [103] In principle, such additions could also be suggested by machine learning algorithms. These Content Aware Studies fall into the still very young field of Computational Creativity. As an example, I would like to mention head fragments of antique portraits that the 3D artist Egor Kraft creates, or rather has created. 104] How absurd the result can be can be observed in this example. The left half of the face shows the Roman emperor Lucius Verus, which has been completed with an unbearded hero's head. If there had been sufficient emperor portraits in the training data, as we scanned them in Göttingen, a different result would certainly have been expected. [105] In the project on the archaeological stylistic analysis of 162 stone masks from Mexico, the aim was to determine masks that cannot be clearly positioned within the three precisely defined styles because they share features of two or more canonical styles. For this purpose, two stylistically distant masks were used as a starting point to artificially generate one hundred virtual 3D models whose shape or "style" lies between the two initial masks. In this way, an ideal-typical, archaeological series was formed, so to speak. It would then be possible to compare each object in the collection with all the other members of the group. [106] This procedure could help to generate an "atlas" of the shape variations expected for such a collection based on feature vectors, which in turn would facilitate the application of a machine learning-based classification method. [107] The international project ArchAIDE took a similar approach to be able to determine pottery fragments on excavations in an automated way. The idea was to scan the sherd with a tablet and immediately match it with the objects in the database. For this purpose, artificial vessels were synthesised from the profile drawings of all types and virtually smashed into ever new sherds. Unfortunately, the result was unsatisfactory. This is because the vessels potted by different potters do not correspond exactly to the type drawings in the identification books. In order to train a neural network on a type, one would need a large number of antique vessels with very different detail characteristics for each type, but these would first have to be digitised. Otherwise, a deep learning algorithm cannot learn what it has to abstract when determining the type. 108] What is geometrically easy to do, however, is to calculate their volume from sections through ancient vessels or the corresponding profile drawings. A simple tool has been put online by the Belgian Centre de recherches archéologiques. [109] In conjunction with 3D models, one could thus reconstruct, for example, the use of Attic cups at the symposium. I have been interested for a long time in the point at which, when drinking the dark red wine, the inner image was visible in the first place. 110] The composition of the inner images of the cups, which are sometimes very small and sometimes hide important details in the lower part of the image, can depend on the hidden part of the picture. So if you are still looking for a topic for a project work, this might be something for you. [111] 3D modelling can also help with the analysis of the original function. In the Etruscanning project, for example, they investigated what the bronze shields found in an Etruscan tomb might have served for. [112] and proposed various hypotheses for reconstruction. Indeed, it is not at all clear whether they were attached to the wheels like hubcaps, or directly to the chariot body. The visualisation of the possibilities now makes it easier for archaeologists to decide on a possible reconstruction. [113] In general, 3D modelling is suitable for any kind of reconstruction, especially when an object is only preserved in its individual parts, from which a plausible antique device must be assembled. [114] In the future, it will be necessary to develop qualitatively and quantitatively transferable methods of shape analysis. Many data-driven applications are based on high-quality shape analysis results, especially shape segmentation and shape comparison. One challenge here is to move from single observations to generally valid results, in terms of automated object mining in computer science and to tried and tested attribution criteria in the object sciences, which also takes into account the physical evidence of pictorial works and artefacts. For humans, it is relatively easy to clarify the essence of forms with a few lines, sketches and abstract shapes. This degree of abstraction is just as important for searching in large amounts of data as it is for typologising and categorising in humanities research. Synthesising and manipulating shapes using shape abstractions as input remains a major challenge for computer science. However, even as a category in image science, the value of shape abstractions in human perception of types, schemas and motifs and interpersonal communication with them has not yet been fully resolved. Data-driven category formation thus offers reason to re-examine established category systems. 115] Data Representations of 3D Shapes Basics of Style Research and Shape Analysis Good Practice Examples of Shape Comparison Digital Shape Analysis Methods of Shape Retrieval and Object Mining [116] Laying and comparing sections through 3D models Perform 3D dimension comparisons with calculation of the Hausdorff distance Sort a set of objects and group them according to type or style. [117] What are the advantages of digital shape analysis of three-dimensional objects? Give an example! Give an application example for tolerance based shape comparison! Why do established methods of computer-assisted face recognition have to fail with historical portraits such as the portraits of Roman emperors? What is meant by style? What are the differences to the concept of type? In what way can 3D data of objects be represented? How might working with collection objects change in the future? What consequences does this have for object-scientific research? [118] I would like to say goodbye to you again with a look at the literature. I wish you a good week and hope you have done well with the content of this lesson.