To finish the project we created a roadmap of our process:
The previous five weeks we have been working on the “Bones” project. Together with coach Dafydd Visscher (later referred to as Dafydd) we have been looking for a universal so-called ‘scaffold’ to help burn victims with reconstructing their damaged ears. At the start, the aim of the project was a bit vague to us. We thought that we were ‘just another group of students’ who would work on something that wasn’t genuinely useful for the VUmc. When we got further into the process, we started understanding the meaning of the project more and more (what does a “general” or “ideal” pattern actually mean in this context?). And more important, it became clearer what our function was within the project. The deeper we got into the project, and thus the more we learned about the subject, the more exciting it became.
The first week was all about researching, and trying to understand the project’s goal. This start was a bit rough since the information was very new to us. We tried researching as much as we could, and tried scheduling meetings with the coaches. It was very nice to look into information other than ‘design-information’ for once. This project is in a way very related to the minor, but concerning the bio-medical part it is completely different. This was refreshing and in good balance with the other project of the minor. As for the following three/four weeks, it was all about finding a pattern, and looking into software that could provide this pattern for us. It was nice for us to work in Amsterdam with a lot of experts on different fields. When we got stuck somewhere in the process, there were always people wanting to help and try different things with us. That’s why we tried going to Amsterdam as much as we can. Also, our coach Dafydd Visscher was always open to questions and suggestions, which made us feel very welcome at the VUmc and which stimulated us to do even more for the project.
The Science Fair felt like icing on the cake. We had our stories straight and we were really proud of what we achieved, so we felt proud presenting our results at the Science Fair. A lot of interested people approached us with interesting questions, which we discussed thoroughly. Some of them were so excited about the project that they kept asking information, reaching subjects which went beyond our acquired knowledge, and raising even more questions along the way, which mattered to the project (e.g. “How is the dissolved biodegradable material later expelled from the body?”). Thanks to the presence of Dafydd all of the “extra” questions could be answered as well as allowing us to go even more into detail with the people who came to listen to our story. Concluding, the entire Science Fair was both a nice presentation of our achievements, and an extra learning moment for all of us.
Even though we managed to finish the project the way we wanted to, with realistic results, the project was very short. Five weeks isn’t enough, given the fact that the first week was focussed on understanding the assignment and biology behind it, and the last week was all about the Science Fair. This leaves three weeks for the actual project, and although this isn’t very long, we are proud to have achieved our initial goals. We all felt a bit sad when the five weeks were over, which is probably a good thing.
Overall the project was very refreshing, especially next to the other minor project which was more focused on construction, mechanics and form-concepts. It was also a very nice project because we had to tackle a challenging problem. We loved working with the people from VUmc, and we think our results are probably the best we could have achieved in this amount of time. We look back at the project with joy and pride, and obviously we are really glad with the grade we got. A special thanks to Dafydd for guiding and helping us, and ofcourse to everyone else involved in the process from VUmc.
Yesterday, the 27th of October, the Science Fair took place at the faculty of Industrial Design at the University of Technology in Delft. At this event we presented our 4 week project on developing a general extracellular matrix of ear cartilage.Above you see a picture of the team members and our PHD coach. From left to right: Patrick Sengalrayan, Feline Hunink, Dafydd Visscher, Jelmer van de Scheur and Femke Maas. Unfortunately Ennio Donders couldn’t be present.
Most important was our final model, which was also 3D printed with a Gypson printer (left image). Furthermore we showed a prototype before the final model (right image) which was a simplified print because the CAD models couldn’t handle the complex inital models.
We finished our project by writing a manual (see also the visual below) on how to create such an extended general extracellular model for ear cartilage, so they can use it also for other bone or cartilage types. Our model will be printed by an american company who can print on a mirco-scale (the above models for example were scaled from 5 mm to 30 cm so they were easy to present) in both collagen as pcl. Afterwards cells are grown in a lab and inserted in the lagunes.
Our process went well but it also had some difficulties which we could solve eventually. You can read and see more about our process in our reflection on the project.
During this week we started working on the materials for the Science Fair. It would of course be excellent if we could show something physical that we made instead of just pictures on a computer screen or printed pictures.
So that is why we started working on a 3D-model that we could print using the 3D printer at the VU Innovation Lab. What we wanted to do for this model is to add the patterns that we found to different layers and somehow connect these. This appeared to be very difficult and needed to be done by hand, but that would take dozens of hours, if not hundreds, so we had to find another way. What we did manage to do is to extrude 2D images of three different patterns and put these on top of each other.
In the final days of the project, we finally managed to find a way to get the 2D patterns into one solid 3D model. The specific steps are explained in detail in “Our Ultimate Scaffold’ (link: http://bones2015.weblog.tudelft.nl/our-ultimate-scaffold/).
Now that we had a 3D model with a closed mesh, we could make a print of it. The model consists of 5 different layers that have been put on top of each other, as explained earlier.
And although a 3D-printed model is the most exciting that we can show, it is also important to show our process and give some general information to understand what we did, so we also started working on posters and a video. For the video we started making a render of the original 3D model of the cartilage. But due to the complexity of this model, the render could not be done in the time that we were at the Innovation Lab and had to be done at home in the weekend.
This week was all about making the created scaffold patterns printable.
The patterns that we created previous week were flat 2D pictures. Cartilage scaffolds are obviously 3d structures. Our first approach to make the created patterns 3D was by image tracing the created scaffold patterns into vectors, and extruding these curves with Rhino. This resulted in the following 3D structure:
The problem with this method was that the lines were extruded but didn’t have a thickness and therefore were not printable. We tried to solve this problem by offsetting the surfaces, but we didn’t have enough computing power to perform this action. When we performed this action on the computers of the VUMC it resulted in a lot of error’s due to intersections and other problems. So it turned out not to be the method we were looking for.
After this we tried a lot of different methods using several software packages like:
- Vgstudio max
All this approaches and use of software did not result in satisfactory results. Then we took a new approach in which we used a combination of Illustrator, Photoshop and Rhinoceros. With this new approach we strongly simplified the pattern by manipulating the pattern in photshop by applying a lot of filters and offsetting the lines. This resulted in this new pattern:
By image tracing this simplified image and cutting the holes away from a solid box in Rhino we were able to produce a higly simplified printable model of one single layer of cartilage. We enlarged this file and send it to the 3D printer to see how it would turn out in real life.
Next to that we also created an image of the parametric ear that has been made during Advanced Prototyping in 2014 with the image of the pattern that we created wrapped around it. This may is not functional yet, but it geves a good impression of what our goal is in the project, which is a printable ear with a cartilage matrix applied to it.
Thursday (8 October) we splitted up the group between Delft and Amsterdam VUmc.
We had an important meeting with an expert on Computer Graphics and Visualization, Elmar Eisenmann (http://graphics.tudelft.nl/~eisemann/ ), at the TUDelft. He took a look at the STL model and concluded the model looked like a 2.5D structure, which will make it possible to take a 2D image (a slice of the STL-model) and multiply it to get a bigger pattern.
Specific algorithms can be used to generate new patterns without boundaries, because they take small parts of multiple pattern-images that fit together best and “glues” those together. To read more about this, here are some of the papers he referred us:
- Fast Texture Synthesis
- Texture Synthesis over Arbitrary Manifold Surfaces
- Solid Texture Synthesis from 2D Exemplars
The group in Amsterdam worked on dividing the mesh in 2D layers and export these as pictures. We also worked with the staff at the 3D Innovation Lab to make a solid mesh to be able to invert it or to find the original scan so we could make a new and hopefully better model. This turned out to be more difficult task than it first seemed and we couldn’t finish it in one day.
The next step is to convert the larger model into a 3D solid, by combining 3 patterns in 3 different planes. Click on the images below to visit the links. For that we need to experiment with different 3D-model programs.
Friday (9 October). Working with layers.
After the layers were processed in illustrator and photoshop we were able to create a new bigger random cartilage pattern. We did this by using software called “Pixplant” that uses the algorithms to create a new pattern from different patterns combined.
Four layers of cartilage combined into one new seamless pattern looks like this:
After some work and discussions, we tried to take and simplify the averaged layers with Photoshop and Illustrator, in order to make 3D models with Rhino. To make it possible for the cells to move into the structure from the sides, we figured that we can make one big solid out of multiple layers taken from different averages, and put them onto each other. The layer averages where taken every 5 slices of the original cartilage (1-5, 6-10, 11-15, 16-20, 21-25, 26-30), finally having 6 different slices one onto each other.
The challenge of this method will be how to print it, probably we need dissolvable/detachable support structure.
The scaffold is printed by a 3D-bioprinter present at the VUmc. Since this printer is quite complicated we need to know more about it so we can prepare our files before printing. We read the printer’s manual & we asked some questions in the meeting with the expert group.
How will the bioprinter eventually print our scaffold?
A PCL outer layer (shell) is printed to prevent contraction of the natural polymer. Daffyd assumes that the cells connect to the PCL layer which makes contraction impossible. The inside matrix consists of the natural polymer, which is a gelatine-like substance with cells.
Bounderies of Bio-Printer
The bioprinter can only read an STL file as a whole, while temperature and other factors have to be controlled per printed layer. Therefore the STL file has to be devided into layers and each layer has to be uploaded to the printer seperately. The printer doesn’t work with STL files but there is a special Bio Cad program for this. The printer works with g-code. What they didn’t try yet is uploading a dwg or dfx file.
How are the images/3D files made of a piece/slice of ear cartillage?
This is done with a laserscanning microscope, which is much more precise than a CT-scan. This is needed because the intercellular matrix of ear cartillage is very small. MRI is also a possibility, however with the laserscanning microscoop the collageen can be distinguished from the elastine.
- Making a solid in Rhino. Trying to simplify the model
One of our first ideas was to invert the 3D model so that we got geometry were the lagunae are instead of the other way around. There a few complications in doing that however. The model we were using was very large, which slowed down our computers dramatically and some functions in the applications we used made our computers crash. Luckily, one of the staff-members at the 3D Innovation Lab could reduce the filesize from around 400 to 50mb without losing a lot of quality.
Another problem was that the model we were given wasn’t a closed mesh so Rhino and the other programs didn’t know what to do. Due to the complexity of the model this isn’t easy to solve.
- Using the programs GOM & 3-Matic on computers of the 3D innovation Lab
At the 3D Innovation Lab at VUmC we got introduced to some new pieces of software. Some of those were free to download and others were licensed so we could only use them on the computers in the lab. Two of the main applications that are being used, are GOM and 3-Matic. Although these are professional pieces of software with a steep learning curve, they did give us some possibilities that we did not have with the software we were initially using. With some help of the staff at the lab we were able to most of the things we wanted with the software.
- Trying out grasshopper
Our goal in this project is to find a pattern in the scaffold of cartilage and we assume this pattern is based on some mathematical law. Nature is very logical and everything works according to laws. A good example would be the golden ratio which can be found in pretty much everything nature creates. A shell doesn’t just accidentally get certain proportions, its growth works according to laws which give the shell its proportions. The same must apply to cartilage. It doesn’t randomly create lagunae (the holes), but they get created according to a law. And it’s this law that we have to find. When we find the pattern, Grasshopper would be an excellent application to create a 3D-model that can be easily modified to get the shape of the patients ear. Since grasshopper is a parametric program, the exact proportions of the ear do not really matter. Theoretically grasshopper could create a ear of infinite size. VUmC already has a parametric model in Grasshopper to create the shape of the ear, so this will have to be combined with a model that creates the scaffold to create the printable ear. We explored some of the features of grasshopper through a tutorial at Digital-Tutors.
- Slicing the STL file
Since we were given a pretty large file, we “sliced” the piece of cartilage in 6 layers to make the files smaller and simpler and to make it easier to further edit the file. Secondly we divided the piece of cartilage in 30 layers, to see if there could be found some sort of pattern or structure through making 2D images of the cross sections, see “creating 2D slices”.
- Creating 2D slices
As mentioned before, one of the ideas was to put multiple cross sections through the 3D-model to get flat 2D pictures of the geometry in that cross section. There are multiple applications to do that, but we found that GOM gave us an excellent result without much work. The application allows you to easily create as many cross sections at any place you want and distribute them equally automatically. You can then control which cross sections you want to be visible, so you can look at one layer of cross sections at the time or several layers at once. We decided to create 30 cross sections to get a good chunk of data, but not drown in it. For every cross section we made a printscreen, resulting in 30 printscreens of cross sections. Now since every cross section has a unique structure, we came up with the idea to take the “average” of multiple cross section images with the help of the program Fiji. Fiji can take two images and take the “average” of these two images. So what we did was give every screenshot a number related to the location of the cross section within the piece of cartilage, and then opening screenshot 1 and 2 and taking the average of these, naming it “average first 2 layers”. Then, opening “average first 2 layers” and the 3rd layers, and taking the average of this layer naming it “average first 3 layers”, and so on. The idea was to make an average at every five layers, so an average of layer 1-5, one of layer 6-10 etcetera. This resulted in six different images of average layers. These six images could then be used to find a pattern, since all six images differ significantly.
After the layers were processed in illustrator and photoshop we were able to create a new bigger random cartilage pattern. We did this by using software called “Pixplant” that uses the algorithms to create a new pattern from different patterns combined. We could only use four of the six images, due to the fact that the upper cross section averages had less material and thus more gaps. We used averages of the bottom 20 cross sections, so images of layers 1-5, 6-10, 11-15 and 16-20.
These four layers of cartilage combined into one new seamless pattern looks like this:
- Setting up a list of programs
The programs we use(d) are:
On the second Friday (2nd of October) there was a discussion on the research we had read and the upcoming plans and scope of the project. Furthermore other core experts of the project were introduced.
Some conclusions: To what extent should our scaffold be similair real ear cartillage? Answer: We assume this can be a simplified scaffold, however it should be tested by printing the scaffolds and letting the cells generate new cartillage wich takes approximately 2 months. Only the cartilage should be mimicked. The perichondrium and skin are other layers. Other parts, like cells will grow from the bioink of the bioprinter.
Ideas – With the research as background, we generated several ideas to edit the original STL Ear cartillage file. Can we for example make a negative which is simpler to edit or can we use a 2D image of the pattern and extrude that later? The next steps are to test different approaches on their feasibillity. We concluded that a formula to find an average pattern would be convienent.
After gaining enough knowledge about (ear) cartilage, 3D and bio printing techniques we could finalize our project proposal. See the objectives and approach, taken from the proposal, below:
- Find the general pattern of ear cartilage
- Recreating matrix/lattice of ear cartilage in simplified form while taking into account cell environment requirements, 3D & bio-printer constraints and strength of the lattice concerning possible future trauma to the ear.
- Final goals: Create a standard model of the ear cartilage lattice and put it in a (helix) shell of the ear, so bio-ink can be printed in between the lattice in future project developments.
- Using Rhino, GOM, 3-matic and/or other available software to simplify the model & create a new model to print with.
- Use Mathlab or/and other pattern recognition software to find a general pattern in the Ear-cartilage STL files ( files available at VUmc 3Dlab Amsterdam, obtained by scanning ears with CT-technology and transferred by experts to STL files).
- Testing (multiple/ different) scaffolds on strength with Finite Element techniques in 3D modelling software (for example GOM, 3-matic and/or Solidworks).
- Doing research about anatomy of ear cartilage and trying to find requirements for the natural environment to stimulate cells to grow new cartilage.
- Using 3Dprinters to print tests of our simplified scaffold and combine them with cells (done by VUmc).
So in the upcoming weeks…we will search for and use a programs that can find general patterns in STL files or images. For that we will contact experts at the TU Delft. In the 3D/Cad-programs we will try out different commands to make the model more simple/ to find a general pattern/to try finite elements on the initial STL file. Furthermore to set-up a list of requirements that our model has to meet we will look at the manual of the VUmc Bio-printer and we have to select a 3D-printer. The bio-printer for example is very complex and can only be used by an expert.
A part of our project takes place at the VUmc which enables us to use more advanced computers and enables us to discuss and cooperate with the experts of the 3D innovation lab.
To proceed with the tests we needed more knowledge about (ear) cartilage and the problem with surgery of damaged ears. Therefore we went through several papers including the topics:
- Different types of ear cartilage and other bone structures & their functions
- Development of the external ear (pinna) (embryology stage)
- Developments & procedure of reconstruction surgery techiques
- 3D techniques used to print ear-scaffolds
- 3D printing used and future advantages in the health sector
- Notions: Scaffold, Lattice, Chondrytes, Perichondrium
- Problem with damaged ears and currently used procedures
Read more about the anatomy of the ear concerning the research focus in About Cartilage.