36C3 preroll music Herald: In the following talk Mr. Bernd Sieker will speak about the crashes and what led to the crashes of the most recent 737 model. He is a flight safety engineer and he also worked on flight safety and he analyzed the plane crashes for a lot of time and a long time. And you have to keep in mind that this 737, although multiple models have been built, can be flown. All models can be flown with the same type rating since 1967, which is one of the many root causes of the issues that led to the disaster that killed 346 people. Let's listen to a Bernd and he'll enlighten us, what else went wrong? applause Bernd Sieker: Yes, thank you very much for the introduction. I see they are not quite as many people as with the Edward Snowden talk, but I'm not disappointed. Aviation safety has always been very important to me and I've done a lot of work on it and I am happy to share my passion with so many of you. Thank you. applause So it's basically the outline of what I'm going to talk about. It's the Boeing 737 Max or seven thirty seven as some may say. I will briefly talk about the accidents, what we knew at the beginning, what went wrong and then what came to light. Later on I will show our causal analysis method that we use very shortly, very briefly and the analysis and overview of the analysis that I did of these accidents. Then talk about the infamous MCAS system, the Maneuvering Characteristics Augmentation System, as it's called, by its full name. Then I'll talk about certification, how certain aircraft certification works in the United States. It's very similar in Europe, although there are some differences. But I'm not going to talk about European details in this talk. So it's mostly about the FAA and aircraft certification across the pond. Some other things and an outlook, how it is going to go on with the Boeing 737 Max. We currently don't know exactly what's going to happen, but we'll see. And if we have time, they have a few bonus slides later on. So the Boeing 737 Max - the star of the show, as you may say, it's the fourth iteration, as the Herald already indicated, of the world's best selling airliner. I think I looked it up just recently. I think there are almost 15,000 orders that have been for the 737 of all the series, the original, the classic, the NG and now the Max. And the Max itself is the fastest selling airliner of all time. So within months, it had literally thousands of orders. It has now almost 5,000 orders. The 737 Max, and all the airlines in the world are waiting for the grounding to be lifted so they can receive and fly the aircraft. So the first accident was last year. It was a Lion Air, an Indonesian flag carrier. Actually, I think the second or third largest Boeing 737 Max customer in the world with a couple of hundred, 250 or something aircraft and it crashed relatively shortly after it entered service. And so we've heard some strange things in the news and on the forums that deal with aviation safety. It seems that there had been uncommanded nose down trim. So the tail plane is moved by an electric motor and it forces the nose of the aircraft down. The pilot can counter that movement with some switches on his control column. And apparently the stick shaker was active during the flight and there were difficulties in controlling the aircraft. We didn't know at the time exactly what it was. And then for the first time, the abbreviation MCAS surfaced and even 737 pilots, even 737 Max pilots, at least some of them said they'd never heard of it. It was a mystery. We later found that actually in some documentation, it was very briefly mentioned that such a system existed, but not exactly why it was there. And I guess Boeing knew and the certification authorities, as it turned out, sort of knew a bit of the story, but not the whole story. But especially people in the West, in the US and in other countries said: Oh, these are just poorly trained Third World pilots. And we expect that. And they weren't completely wrong. Lion Air has a particularly bad safety record. And it wasn't unknown to aviation safety investigators. There have been a number of crashes with Lion Air. So in the beginning, we thought, OK, maybe it's a fluke, it's a one off or maybe it's caused by poor maintenance or bad pilots or whatever. So several people, on the other hand, already began worrying because some flight data recorder traces became public. And there was some very strange things which we will see shortly. And then only a few months later, the second aircraft of exactly the same type and the same variant, Boeing 737 Max 8, also crashed. And you can see maybe on the picture on the left, it left a rather big crater. It really dove into the earth quite fast. It turned out, I think, about between seven and eight hundred kilometers per hour. So, so really fast and not much left. Not much was left. I think the biggest parts were about this size, I guess. So all small pieces of debris and the engine cores, which are a bit bigger. And from that as well, flight data recorder traces became public. The recorders had survived at least the memory in them and were readable. So we finally found out something and found some similarities, some rather disturbing similarities. We come to that in a moment, but I'll talk a little bit about the Boeing 737 family in general. So there were four, as I said, models. That was the original, which had narrow engines under the wings. Not a lot of room between the ground and the engines, but it looked quite normal. You could say it was one of the first short- haul airliners with under slung engines, under the wings and then new high bypassed turbo fire engines entered the market, which were much more fuel efficient. We're talking about maybe some 15 to 20 percent lower fuel consumption. So it was a big deal. And the Boeing 737 was reengined and became known as the classic, bigger engines, but still mostly analog mechanical instruments. And it was basically the same as the original, instead that it had some bigger engines and they had to shape the cowling a little differently to accommodate the bigger engines. But more or less, it worked for a while. And then as airlines demanded more modern avionics, so the cockpit electronics in aircraft, the next generation was conceived. It also got a new wing, new winglets, which again saved a lot of fuel. It had basically the same engines, except that the engines now were also computer controlled by what we call FADEC full authority, digital engine control. And Boeing said, well, that's probably going to be the last one. And in the next few years, we are going to develop an all new, short and medium haul single aisle aircraft which will be all new and super efficient and super cheap to operate - all the promises that manufacturers always make. In the meantime, Airbus was becoming a major player with the A320. It was overall a much more modern aircraft. It had digital fly by wire. It always had digitally controlled engines. It had much higher ground clearance. So it was no problem to accommodate the larger engines in the A320. And Airbus then announced that it was going to reengine the A320. And for the A320, that was the first time it got new engines. It for a long time it had you had the choice of two types of engines for the A320 And then they said, we're going to install these new super efficient engines, which brought with it another optimization of fuel consumption. That was another 15 percent fuel saved per mile traveled something on the order of that. So it was a huge improvement again. And many Airbus customers immediately ordered the so-called A320neo and some Boeing customers also thought, well, this one is going to consume so much less fuel that we might consider switching to Airbus, even though it's a major hassle if you have fleet entirely consisting of Boeing aircraft, if you then switch to Airbus, it's a huge hassle and nobody really wants that unless they're really forced to. But the promised fuel savings were so big that companies actually considered this and lots of them. And so Boeing said we need something very quickly, preferably within two years I think. For airline development, that's very, very, very, very quickly. And they said, well, scrap all the plans about the new small airliner. We're going to change the 737 again. And now the new engines, were going to be bigger, again. And so actually, there was no ground clearance to move them in the same way as on the on the NG. So there to modify the landing gear, to mount the engines even further forward and higher. And the engines were bigger. But the engines were, on the whole, they were very good new development. The same type of engines that you could get for the new Airbus - CFM international. And so we decided to make the Boeing 737 4th generation and call it "the Max".So when we analyze accidents, we use a causal analysis method called Why-Because analysis. And we have some counterfactual tests which determines if something is a cause of something else. We call it a necessary causal factor. And it's very simple. A is a causal factor of B, if you can say had A not happened, then B would not have happened either. So, I mean, you need to show for everything that there is a causal relationship and that all the factors that you have found actually sufficient to cause the other event. So you can probably not read everything of it, but it's not really important. This is a simplified graph and I will show the relevant details later.And this is the analysis that I made of these accidents. And you can see it's not a simple tree; as computer scientists, many of you are familiar with trees and this is just a directed graph and it can have branches and so on. And so some things are causal influence, causal effect of several different things. So some of the factors actually have an influence on multiple levels. For example, the airspeed influences the control forces and it also influences the time the crew had to recover the aircraft before impact with the ground. So these are some of the things that I will look at in a bit more detail. So here is one of them: Uncommanded nose down trim. So what happened apparently on these accident flights was that you can see it in the flight data recorder traces. I don't know. Can you see the mouse pointer? Here, that's the blue line. And that is labeled trim manual. And there's the orange line that is labeled Trim Automatic. And if they have, do displacement to the bottom, that means that the aircraft is being trimmed nose down, which means in order to continue to fly level, you have to pull the control column with more force towards you. And what you can see is in the beginning, there are a few trim, trim movements. And on this type, they are expected it has an automatic trim system for some phases of flight which trims the aircraft to keep it flying stable. And then after a while, it started doing many automatic nose down trim movements. Each of these lasts almost 10 seconds and there is a pause between them. And in every case, the pilots counter the nose down trim movement with the nose up trim movement on the control yoke. There are switches that you operate with your thumb and you can trim the aircraft that way and change the control forces and cause the aircraft nose to go up or down. So for a very long time, this went on: The computer trimmed the aircraft nose down, the pilots trimmed the aircraft nose up, and so on. Until at the very end, you can see that the trim, the nose up trim movements that the pilots made, become shorter and shorter. And this line here, it says pitch trim position. That is the resulting position of the trim control surface, which is the entire horizontal stabilizer on the aircraft. And it moves down and it doesn't really go up anymore because the pilot inputs become very short. And that means the control forces to keep the aircraft flying level become extremely high. And in the end, it became uncontrollable and crashed, as you can see here. So the pilots, for various reasons, which I will highlight later, the pilots were unable to trim the aircraft manually and the nose down trim persisted and the aircraft crashed. And this is only the graph of one of the accidents. But the other one is very similar. And so that's what we see. There is a known system, which was already known before on the Boeing 737. I think it's available on all the old versions as well, which is called the speed trim system, which in some circumstances trims the aircraft automatically. But the inputs that we see, the automatic trim inputs don't really fit the so-called speed trim system. And so for the first time, we hear the word MCAS. And we'll talk a bit more about what made the Boeing 737 different from all the previous models. And that is the bigger engines. As I said, the engines were much bigger. And to achieve the necessary ground clearance, they had to be mounted further forward. And there are also a lot bigger, which means at high angles of attack, when the aircraft is flying against the stream of the oncoming air at a higher angle, these engine cells produce additional lift in front of the center of gravity, which creates a pitch up moment. And the certification criteria are quite strict in that and say exactly what the forces on the flight controls must be to be certified. And due to the bigger engines, there was some phases or some angles of attack at which these certification criteria were no longer met. And so it was decided to introduce a small piece of software which would just introduce a small trim movement to bring it in line with certification criteria again. And one of the reasons this was done was probably so the aircraft could retain the same type certificate as was mentioned in the introduction. So pilots can change within one airline, between the aircraft, between the 737 NG and the 737 Max. They have the same type certificate. There's a very brief differences training, but they can switch even in line operations between the aircraft from day to day. And another reason. No other changes were made. Boeing could, for example, have made a longer main landing gear to create additional ground clearance to move the engines in a more traditional position, that would have probably made it more aerodynamically in line with certification criteria. I hesitate to say the word "to make it more stable" because even as it is, the Boeing 737 Max is not inherently aerodynamically unstable. If all these electronic gimmicks fail, it will just fly like an airplane and it is probably in the normal flight envelope easily controllable. But to make big mechanical changes would have delayed the project a lot and would have required recertification and what instead could be done with the airframe essentially the same. The certification could be what is known as grandfathered: so it doesn't need to fulfill all the current criteria of certification, because the aircraft has been certified and has been proven in service. And so only some of the modifications need to be recertified, which is much easier and much cheaper and much quicker. So this is one of the certification criteria that must be fulfilled. It's even though I have removed some of the additional stuff that doesn't really add anything useful, it's still rather complicated. It's a procedure that you have to do where you slow down one knot per second. And the stick forces need to increase with every knot of speed that you lose and things like that. And it says it this stick force versus speed curve may not be less than one pound for each six knots. And it's quite interesting, if you look at the European certification criteria, is that they took this exact paragraph and just translated the US units into metric units, but really calculated the new value. So the European certification have now very strange values like, I don't know, 11.79 kilometers per hour, per second or something like that. It's really strange. So you can see where it comes from. But they said we can't have knots even though the entire world except Russia and China basically flies in knots, even Western Europe. But the criteria in the certification specification need to be in kilometers per hour. Well, I would have thought that you would even - if you do the conversion, you would use meters per second, but it used kilometers per hour for whatever reason. So due to the aerodynamic changes that were made, the Max did not quite fulfill the criteria to the letter. So something had to be done. And as I said, mechanical redesign was out of the question because it would have taken too long, would have been too expensive, and maybe would have broken the type certificate commonality. So they introduced just this little additional software in a computer that also existed already. And so it measures angle of attack, it measures airspeed and a few other parameters, flap configuration, for example, and then it applies nose down pitch trim as it sees fit. But it has a rather interesting design from a software engineering point of view. Can you read that? Is that... They are flight control computers. And one part of this flight control computer, one additional piece of software, is called the MCAS, the Maneuvering Characteristics Augmentation System. And the flight control computer actually gets input from both angle of attack sensors. It has two, one on each side for redundancy, but the MCAS algorithm only uses one of them, at least in the old version. In the new version, it will probably use both if it ever gets recertificated. And then if that angle of attack sensor senses a value that is too high, then it introduces nose down trim and it may switch between flights between the left and the right sensor. But at any given time for any given flight, it only ever uses one. So what could possibly go wrong here? Here we can see what went wrong. It's the same graph as before, and I may direct your attention to this red line that says angle of attack indicated left and the green line which says angle of attack indicated right. So that is the data that the computer got from the angle of attack sensors. Both are recorded in the data recorder, but only one is evaluated by the MCAS. And you can see here's the scale on the right. You can see that one is indicating relatively normally around zero, a bit above zero, which is to be expected during takeoff and climb. And the red value is about 20 degrees higher. And of course, that is above the threshold at which the MCAS activates. So it activates. Right. And apparently in the old version of the software, there were no sanity checks, no cross checks with other air data values like airspeed and altitude or other things. And it would be relatively easy to do. Not quite trivial. You have to get it right in these kinds of things which influence flight controls, but nothing too fancy. But apparently that was also not done. So the MCAS became active. So how could it happen? And it's still to me, a bit of a mystery how it could actually get so far that it could be certified with this kind of system. And the severity of each failure, the possible consequences have to be evaluated. And the certification criteria specify five severities: catastrophic, hazardous, major, minor and no safety effect, and that doesn't have to be analyzed any further, but for catastrophic failures, you have to do a very, very complex risk assessment and see what you can do and what needs to be done to bring it in line, to make it either mitigate the consequences or make it so extremely improbable that it is not going to happen. So here are the probabilities with which the certification criteria deal and its different orders of magnitude. There are usually two orders of magnitude between them. It's from a probability of 1 times 10 to the minus 5 per hour to 1 times 10 to the minus 9 for operating hour. And this is the risk matrix. Many of you are probably familiar with those. And it basically says if something is major, then it may not happen with a probability of probable. And if its catastrophic the only probability that is allowed for that is extremely improbable. Which is less than once in a billion flight hours. Right. And to put that into perspective, the fleets with the most flight hours to date, I think, are in the low hundreds of millions of flight hours combined. So we're still even for the 737 or the A320. We're still quite far away from a billion flight hours. So you might have expected perhaps one of these events because statistical distribution being what it is, the one event might happen, of course, and but certainly not two in less than two years. And quite obviously, the severity of these failures was catastrophic. I think there's no - there's no discussion about that. And here's the relevant part, actually, about flight controls and the certification criteria, which was clearly violated. It says the airplane must be shown to be capable of continued safe flight for any single failure. Without further qualification, any single system that can break must not make the plane unflyable or any combination of failures not shown to be extremely improbable - and extremely improbable is these 10 to the minus 9 per hour. And this hazard assessment must be performed for all systems, of course, and severity must be assigned to all these. And the unintended MCAS activation was classified as major. And let's briefly look at that. What's major? Reduction in capability, maybe some injuries, major damage. So nothing you can just shrug off, but certainly not an accident with hundreds of dead. So and therefore, there are some regulations which say which kinds of specific analysis you have to do for the various categories. And for major no big failure modes and effects analysis FMEA, was required. And these are all findings from the Indonesian investigation board. And they're all in the report that is publicly downloadable. In the final version of the slides, I'll probably put some of the sources and links in there so you can read it for yourselves. It's quite eye opening. So only a very small failure in failure analysis was made, comparatively small. It probably took a few man hours, but not as extensive as it should have been for the event had it been correctly classified as catastrophic. And some of these things that could happen were not at all considered, such as large stabilizer deflection. So continued trim movement in the same direction or a repeated activation of the MCAS system, because apparently the only design of the MCAS system that the FAA saw was limited to a 0.6 degree deflection at high speeds and to one single activation only. And that was changed. And it is still unclear how that could happen. It was changed to multiple activations, even at high speed. And each activation could move the stabilizer as much as almost 2.5 degrees. And there was no limit to how often it could activate. And what was also not considered was the effect of the flight characteristics caused by large movements of the stabilizer or movement of the stabilizer to the limit of the MCAS authority. The MCAS doesn't have authority to move the stabilizer all the way to the mechanical stop, but only a bit short of that, much more than the manual electric trim is capable of trimming the airplane on the aircraft. You can always trim back with a manual electric trim switches on the yoke, but you cannot trim it nose down as far as MCAS can. So that's quite interesting. That was not considered. What was also not considered, at least it wasn't in the report apparently that the Indonesian agency had seen, was that flight crew workload increases dramatically if you have to pull on the yoke continuously with about, let's say, a force equivalent of 40 kilograms of 50 kilograms continuously, otherwise if you let go, you're going to go into a very steep nosedive. And at that short, it is at a low altitude that they were they would not have been able to recover the aircraft. And in fact, they weren't. What was also not considered was an AOA sensor failure in the way that we have seen it in these two accidents, although apparently they those had different causes. The effect for the MCAS was the same, that one of the sensors showed a value that was about 22 and a half degrees too high. And that was not considered in the analysis of the MCAS system. So I hope that is readable. That is a simplified state machine of the MCAS system. And what we can see is that it can indeed activate repeatedly, but only if the pilot uses the manual electric trim in between. It will go into a dormant state if the pilot trims manually with the hand wheel or if the pilot doesn't use the trim at all, it will go dormant after a single activation and stay that way until electric trim is used. So that's the basic upshot of this state machine. So when the pilot thinks he's doing something to counter the MCAS and he's actually making it worse. But this isn't documented in any pilot documentation anywhere. It will probably be in the next way. If it's still working like that. But so far it wasn't. So Boeing was under a lot of pressure to try to sell a new, more fuel efficient version of their 737. And so I can't say for sure how it was internally between the FAA and Boeing, but it's not unreasonable to assume that they were under a lot of pressure from management to accelerate certification and possibly take shortcuts. I can't make any accusations here, but it looks that not all is well in the certification department between Boeing and the Federal Aviation Authority. So originally, the idea, of course, is the manufacture builds the aircraft, analyzes everything, documents everything, and the FAA checks all the documentation and maybe even looks at original data and maybe looks at the physical pieces that are being made for the prototype and approves or rejects the documentation. There is already a potential conflict that is not there in most other countries because they have separate agencies. But the FAA has a dual mandate. It is supposed to promote aviation, to make it more efficient, but also to ensure aviation safety. And there may be conflicts of interests, I think. So here's what this certification has been up until not quite sure, 10, 15 years ago. So the FAA, the actual government agency, the Aviation Authority, appoints a designated engineering representative. The DER is employed and paid by Boeing, but is accountable only to the FAA. And the DER checks and documents everything that is being done. There's usually more than one, thatt for simplicity's sake, let's say. And the DER then reports the findings and all the documentation, all the low level engineering and analysis documentation that has been done to the FAA, and the FAA signs off on that or asks questions and visits the company and looks at things and makes audits and everything like that. And so that usually has been working more or less and has certainly improved the overall safety of airliners that have been built in the last decades. And this is the new version. And the person is now not called DER, but it's called AR, the authorized representative, is still employed and paid by Boeing. That hasn't changed, but is appointed by Boeing management and reports to Boeing management. And the Boeing management compiles a report and sends that to the FAA and the FAA then signs off on the report. They hopefully at least read it, but they don't have all the low level engineering details readily available and only rarely speak to the actual engineers. So anyone seeing a problem here? Well, you have to say that most aircraft that are being built have been built in the last years aren't really terrible. Right. The 787 is a new aircraft. The 777 has been one of the safest aircraft around, at least looking at the flight hours that it has accumulated. So it's not all bad, but there's potential for real, really bad screw ups. I guess. There's another factor maybe that I've briefly mentioned is that the Boeing 737, even in its latest version, is not computer controlled. It's not fly by wire, although it has some computers as we have seen, that can move some control surfaces. But mostly it's really, it really looks like that. I think that's an actual photo from a 737 has some corrosion on it. So it's probably not a max an older version, but it's basically the same, which is also why the grandfathering certification still works. So it's all cables and pulleys and even if both hydraulic systems fails - so, yes, it is hydraulically assisted, the flight controls - but if both hydraulic systems fail with the combined forces of both pilots, you can you can still fly it and you can still land it. That usually works, except when it doesn't. And the cases where it doesn't work are when the aircraft is going very fast and has a very high stabilizer deflection. And this is from a video some of you may have seen there, it's from Mentour Pilot. And he has actually tested that in a full flight simulator, which represents realistic forces on all flight controls, including the trim wheel. You can be in the center console under the thrust levers, there are these two shiny black wheels and they are the trim wheels. You can move them manually in all phases of flight to trim the aircraft. If electric trim is not available. Pilot: in the normal trim system would not do this. OK. It would require manual trimming to get it away from this. That's fine, it's fine, trim it backwards. Trim it backwards again Bernd: So now he is trying to trim it nose up again after he has manually trimmed it nose down because the normal electric trim system cannot trim it so far nose down. They have to do it manually. And now he is trying to trim it back nose up from the position which is known from the flight data recorder that it was in the accident flight and is trying to trim it manually because some people said: "oh, turn off the electric trim, the electric trim system and trim it manually. That will always work." And they're trying to do that. And it has representative forces to the real aircraft. Copilot: Oh my god. heavy breathing Pilot: Ok, pause the rec... Bernd: and you can see that the pilot on the left, the captain, can't even help him. In theory, both could turn the crank at the same time. And they have a handle on both sides because he has to hold the control column with all his force. So you can't let go. He must hold it with both arms. Otherwise, it would go into a nosedive immediately. And this is the physical situation with which the pilots were confronted in the accident flight. And he now says: "press the red button in the simulator." So end the simulation because it's clear that they're going to crash. So there is another thing that came that came up after the accidents and 737 pilot said: "oh, it's just a runaway trim, runaway stabilizer trim, there's a procedure for that and just do the procedure and you'll be fine." Well, runaway stabilizer trim is one of the emergency procedures that is trained ad infinitum. Right. That's something that every 737 pilot is aware of because there are some conditions under which the trim motor always gets electric current and doesn't stop running. That just happens occasionally, not very often, but occasionally. And every pilot is primed to recognize the symptoms. Oh, this is one of a runaway stabilizer. And you turn off the electric motors for the stabilizer trim and trim manually and that'll work. But if you look at what are the actual symptoms of runaway stabilizer, it says uncommanded stabilizer trim movement occurs continuously. And MCAS movement isn't continuously, MCAS trim movement is more like the speed trim system, which occurs intermittently and then stops and then trims again for a bit and then stops again. So most pilots wouldn't recognize this as a runaway trim, because the symptoms are very different. The circumstances are different. So I guess some pilots might have recognized that there's something going on with the trim that is not right and will have turned it off. But some didn't, even though they know they all know about runaway stabilizer. And yeah, that's the second file that I have. loud rattling noise So that's the sound. The stick shaker makes on a Boeing 737. And now imagine flying with that sound all the while shaking the control column violently, flying with that going on for an hour. And that's what the crew on the previous flight did. They flew the entire flight of about an hour with a stick shaker going. I mean, that's quite that's quite interesting because the stick shaker says your wing is about to stall. Right. But on the other hand, they knew they were flying level. They were flying fast enough. Everything was fine. The aircraft wasn't about to stall because it was going fast and. Right. So from an aerodynamics perspective, of course, they could fly the airplane because they knew it was nowhere near a stall. But still, I think in most countries and most airlines, they would have just turned around and landed again and saying the aircraft is broken, please fix it. Something is wrong. But yeah. So the stick shaker is activated by the angle of attack reading on each side and it sticks out mechanically coupled of both of them will shake with activation from either side. So is it going to fly again? It's still somewhat of an open question, but I suspect that it will because it's it's hard to imagine that letting these 460 airplanes or some something like that that have been built sometimes sitting around on an employee parking lots like here, just letting them be scrapped or whatever. I don't know. Almost 5000 have been ordered. As I said, neither airlines nor Boeing will be happy. But it's not quite clear. It's not yet being certified again. So it's still unairworthy. So there's another little thing, certification issues with new Boeing aircraft. Reminded me of this. Have you ever seen that? So battery exhaust, which the aircraft has a battery exhaust? I mean, what did you do with that? Does anybody know? Yeah, of course some know. Yeah. Boeing 787 Dreamliner. Less than two years after introduction. Now, after entering the service, actually had two major battery fires. They have two big lithium ion batteries. Lithium, lithium, cobalt. I think, not sure. The one that burns the brightest. laughter Bernd: Because they wanted the energy density, really, and that wasn't available in other packages. If they had used nickel cadmium batteries instead, they would have been like 40 kilograms heavier for two batteries. That's almost a passenger. So yeah, they were onboard fires. And if you ask pilots what's your worst fear of something happening in flight, they'll say: flight control failure and fire. So you don't want to have a fire in the air, absolutely not. And one of the fires was actually in-flight with passengers on board. One was on the ground shortly after disembarking and the lithium ion batteries, because they are unusual and a novel feature, as it's called, have special certification conditions because they are not covered by the original certification criteria, and it says here: Safe cell temperatures and pressures must be maintained during any foreseeable condition and during any failure of the charging system, not shown to be extremely improbable... extremely remote, sorry, and extremely remote is actually two orders of magnitude more frequent than extremely improbable. Extremely remote is only less than once every 10 million flight hours. But I think the combined flight hours for the 787 at that time were, not quite sure, maybe a few hundred thousand at most. So and also happened two times. There was not really not really fun. And then it says no explosive or toxic gases emitted as the result of any failure may accumulate in hazardous quantities within the airplane. I think they've neatly solved the third point by putting the battery in a stainless steel box, really thick walls maybe, I don't know, eight millimeters or something like that. And piping them to this hole in the bottom of the aircraft. So the gases cannot accumulate in the aircraft, obviously. So, yes. And with that, I'm at the end of my talk and there's now, I think quite some time for questions. Thank you. applause Herald: Extremely punctual, I have to say. Thank you for this interesting talk. We do have the opportunity for quite some questions and a healthy discussion. Please come to the microphones that we have distributed through the hall. And while you queue up behind them: Do we have a question from the Internet already? Dear signal Angel. Is your microphone working? Signal Angel: No. Herald: Yes. Signal Angel: Yes. Do you think extensive software tests could have solved this situation? Bernd: Software tests in this case, perhaps? Yes. Although software tests are really a problematic thing because to test software to these extreme reliability is required. You really have to test them for a very, very, very, very long time indeed. So to achieve some confidence, they have 99 percent that a failure will not occur in, say, 10 million hours, you will have to test it for 45 million hours. Really. And you have to test it with the exact conditions that will occur in flight. And apparently nobody's thought of an angle of attack failure, angle of attack sensor failure. So maybe testing wouldn't have done a lot in this case. Herald: Thank you. Microphone number four. Mic4: Yes. Thank you for the talk. I've got a question concerning the grounding. So what is your view that the FAA waited so long until they finally ground the aircraft a week after the Chinese started with grounding. Bernd: Yes, that's a good point. And I think it's an absolute disgrace that they waited so long. Even after the first crash. They made an internal study and it was reported in the news some some weeks ago and estimated that during the lifetime of the 737 max, probably around 15 aircraft would crash. So I say every two to three years, one of them would crash and they still didn't ground it and waited until four days after the second accident. Yes, it's a shame, really. Herald: Thank you. Microphone number seven, please. Mic7: Thank you for your talk. I have a question regarding the design decision to only use one AOA sensor. So I've read that Boeing used the MCAS system before on a military aircraft and that used both sensors. So why was that decision made to downgrade? Bernd: Yeah, that's a good question. I'm not aware of that military system. If that was really exactly the same. But if that's the case, yes, that makes it even stranger that they chose to use only one in this case. Yes. Thank you. Herald: Okay, Microphone number two, please. Mic2: Yeah. Thank you for your talk. So how do you actually test these requirements in practice? So how you determine in practice if something is likely to fail every ten to the minus nine as opposed to every ten to the minus eight? Bernd: No, that's that's obviously practically completely impossible. You can't. As I said, if you want to have a reasonable confidence that it's really the error rate is really so low, you'd have to test it for four and a half billion hours in operation, which is just impossible. What instead is done: there are some, industry standards for aviation that is DEO178 currently in revision C, and that says if you have software that if it fails, may have consequences of this severity, then you have to use these very strict, very formal methods for developing the software, like doing very strict and formal requirements analysis specification in a formal language, preferably. And um, if possible, and some some companies actually do that, formally prove your source code correct. And in some languages that can be done. But it's it's very, it's a lot of effort. And that's how this should be done. And this software obviously should have been developed to the highest level according to the DEO178, which is level A and quite obviously it wasn't. Herald: Thank you. Signal Angel, please. The next question from the Internet. Signal Angel: The talk focused most on MCAS, but someone noted that the plane was actually designed for engines below the wings and the NG model, so the one before, already had problems of the wing mounts and engine mounts. Do you think there will be mechanical problems with Max, too? Bernd: I'm not sure there were really mechanical problems. There were aerodynamic problems. And apparently. Well, I'm sure they have tested the NG to the same standards, to the same certification standards, because obviously there were aerodynamic changes even with the NG. And the NG apparently still fulfilled the formal criteria of the certification. There are some acceptable means of compliance and quite specific descriptions, how you test these stick forces versus airspeed. And as far as I know, the NG just fulfilled them. And the Max just didn't. So for the Max, something was required, although even the classic, which basically at the same engine as the NG. Even the classic had some problems there. That's where the speed trim system was introduced. And so it has a similar system and actually the MCAS is just another little algorithm in the computer that also does the speed trim system. Herald: Please stay seated and buckled up until we reach our parking position. No. We are still in the Q&A phase. Please stay seated and please be quiet so we can enjoy all of this talk. And if you have to have to leave, then be super quiet right now. It is a way too loud in here, please. The next question from microphone number one. Mic1: So considering lessons learned from this accident, has the FAA already changed the certification process or are they about to change it? Or on what about other agencies worldwide? Bernd: The FAA is probably going to move very slow. And I'm not aware of any specific changes yet, but I haven't looked into too much detail in that. Other certification agencies work somewhat different. And at least the EASA in Europe and the Chinese authorities have already indicated that in this case they are not going to follow the FAA certification, but going to do their own. And until now, it was usually the case that if the FAA certified the airplane, everybody else in the world just took that certification and said what the FAA did is probably fine and vise versa. When the EASA certified a Boeing airplane, then the FAA would also certify it. And that is probably changing now. Herald: Thank you. Microphone number 3. Mic3: So, hi. Thank you for this talk. Two questions, please. Were you part of an official investigation or is this your own analysis of the facts? Here's the other one. I heard something about this software being outsourced to India. Can you comment on that, please? Bernd: The first one: no, this is my own private analysis. I have been doing some accident analysis for a living for a while, but not for any official agency, but always for private customers. And about outsourcing to India, I'm not quite sure about that. I've read something like that. And what I've read is that it was produced by Honeywell. I think. I may be wrong about that, but I think it was Honeywell. And who the actual programmers were sitting. If it's done properly, according to the methodologies prescribed by DO178 and fulfilling all those requirements, then where the programmer sit is actually not that important. And I don't want to deride Indian programmers, and I think if it's done according to specification and analyzed with study code analyses and everything else vis a vis the specification, then that would also be fine, I guess. But the problem is not so much really in the implementation, but in the design of the system, in the architecture. Herald: Thank you. Microphone number 5 please. Mic5: Hello. I may go to your presentation wrong, but for me, the real root cause of the problem is the competition and high deadline from the management. So the question for you is: is there any suggestions from you that process could be, I dunno, maybe changed in order to avoid the bugs in the software and have the mission critical systems saved? Bernd: Yeah. So we don't normally just talk about THE cause or THE root cause, but there are always several causes. Basically you can say depending on where you stop with the graph - where is it? - where you stop with the graph all the leaves on the graph are root causes and but I've stopped relatively early and not not I'm not gone into any more detail on that, but yeah. The competition between Airbus and Boeing, obviously it was a big factor in this. And I don't suppose you do suggest that we abolish competition in the market. But what needs to be changed, I think, is the way certification is done. And that requires the FAA reasserting its authority much more. And that will probably require a lot more personnel with good engineering background, and maybe that would require the FAA paying better wages. So I don't know, because currently probably all the good engineers will go to Boeing instead of the FAA. But the FAA dearly needs engineering expertise and lots of it. Herald: Thank you. The next question we hear from microphone number 4. Mic4: Hi. Thank you for the talk. I've heard that there is - I've heard - I've read that there's a version of the 737 Max 8 that did allow for a third airway sensitivity present that served as a backup for either sensors but that this was a paid option. And I have not found confirmation of this. Do you know anything about this? Bernd: No, I'm not aware of that as a paid option. There was something about an optional feature that was called a safety feature, but I can't exactly remember what that was. Maybe it was and angle of attack indicator in the cockpit that is available as an option, I think, for this 737 for most models, because the sensor is there anyway. As for a third AOA sensor, I'd be surprised if that was an option because that is a major change and requires a major change to all the system layout. Then you'd need an additional a data inertial reference unit, which is a big computer box in the aircraft of which there are only two. And that would've taken a long, long time in addition to develop. So I'm skeptical about that third angle of attack sensor. At least I've not heard of it. Herald: Thank you. Signal angel, do we have more from the internet? Please one quick one. Signal angel: If you need a quick one, would you ever fly with a 737 Max again if it was ever cleared again? applause Bernd: I was expecting that question. And actually I don't have an answer yet for that. And that maybe would depend on how I see the FAA and the EASA doing the certification. I've seen some people saying that the 737 Max should never be recertified. I think that it will be. And I look at it in some detail, seeing how the FAA develops and how the EASA is handling it. And then maybe. Yes. Herald: Great. Okay, in that case, we would take one more very short question from microphone number 5. Mic5: Do you know why the important AOA sensor failed to give the correct values? Bernd: There are some theories about that, but I haven't investigated that in any more detail now. There were some stories that in the case of the Indonesian, the Lion Air, that it was actually mounted or reassembled incorrectly. That would explain why there was a constant offset. It may also have been somebody calculated that it was actually, exactly - if you look at the raw data that is being delivered on the bus -, there was exactly one flipped bit, which is also a possibility. But I I don't really know. But there were some implications in the report. Maybe I have to read that section again from the Indonesian authorities about substandard maintenance, as it is euphemistically called. Herald: OK. We have two more minutes. So I will take another question from microphone number 1. Mic1: Hey, I would have expected that modern aircraft would have some plug, physical plug, hermetic one that would disconnect any automated system. Isn't this something that exist in our planes today? Bernd: Now, and especially modern aircraft can't just disconnect the automatics, because if you look at modern fly by wire aircraft, there is no connection between the flight controls and the control surfaces. There's only a computer and the flight controls that the pilots handle are only inputs to the computer and there's no direct connection. That is true for every Airbus since the A320, for every Boeing since the triple 7, so the triple 7 and the 787 are totally 100 percent fly by wire. Well, I think 95 percent because there's one control service that is directly connected, one spoiler on each side. But basically, there's there's no way. And so you have to make sure that flight control software is developed to the highest possible standards. Because you can't turn it off, because that's everything. That's, Well, let me put it this way: On the fly by wire aircraft, only the computer can control the flight, the flight control surfaces know. So I just hope that it's good. Herald: Think about that when you next enter a plane. And also, please give a big round of applause for our speaker Bernd. applause 36c3 postroll music Subtitles created by c3subtitles.de in the year 2020. Join, and help us!