Dialogs for computer-human-teamwork

The recent failure of the russian rocket carrying Cosmos 1 as well as other failures like that of the Delta4 Heavy seem to be an argument to enable more teamwork between computers and humans correcting, assisting and controlling each other - already discussed a little bit in the thread piloted flights, automatic flights - hybrid? .

I already mentioned dialogs between the computers aboard rockets and vehicles today. I suppose that engineers may feel terrified by that idea and one example I am temptated to use here may cause "special" arguments.

So I'll simply begin with the checks going on aboard a rocket or vehicle.

As far as I understand it the computers aboard have the exclusive control of the engine including burn time and so on. The ground stations get telemetry and other data simply to watch the things but can't take over control. When a failure like in the example of the Delta 4 Heavy occurs the data must be analysed and it takes at least one day to detect the error.

So why not provide a dialog for checking? The data the computer gets from the sensors can be sent to the ground station additional real time and I assume that is really done already since long - but why not add something very simply to this given the current cheap technologies? If the sensors providing data to the computer indicate propellant running empty - why not add a simple webcam sending an image of the whole inner of the tank to the ground station? The station then could compare the image to the data the computer is getting from the sensors.

The dialog could be a very simple one - the real time video enriched by the propellant data got by the sensors.

Additionaly sound data could be used perhaps.

This should enable to detect deviations between data and real situation very quickly.

To what components, parts and elements could something like this applied too?

And now the example which may cause "special" arguments - Peter, please don't use such "special" arguments like "fantasy" and so on. "Star Trek" is using one special control method - the engineers can see the reaction pattern of matter to antimatter nearly directly by looking to the cylinder it's going on in. Peter - of course that's nonsense, but the use of a webcam is similar to that and currently I have no informations which say that it can't be done. And if it is impossible today then I am trusting the engineers that they can find a way.

So what about such dialogs?



Dipl.-Volkswirt (bdvb) Augustin (Political Economist)

The real problem is not lack of data, it is reaction time. Existing sensors and telemetry are as good or better than webcams and so on. The real problem is that many complex decisions must be made in fractions of a second, faster than any human could. That is why control is given to computers. As to the specific idea of a webcam in the fuel tank, why bother? You know the fuel is exhausted when the engine stops, as in the airplane example I mentioned on the off topic area.

Obviously this wasn't the case reagrding the Delat 4 Heavy - the sensors indicated that propellant was exhausted although it was NOT. The data provided by the senors were wrong - and the cause were bubbles.

It took one day at least to detect that. If there would have been a webcam simply providing data to the ground station a simply comparison of the data - number 0 litres of propellant or something like that - to the videos showing propellant but bubbles in it would have indictaed and shown AT ONCE that there were errors in the data causing errored engine shutdown.

So was quite wrong in this case - the engine stopped but there still was fuel.

This doesn't have to do with complex decisions and fractions of seconds - I know of it and of it being the reason of giving control to the computers but it should be considered later.

A dialog providing something that simple as the comparison of a video to sensor data could save a lot of time when the causes of an errored engine shutdown has to be examined.



Dipl.-Volkswirt (bdvb) Augustin (Political Economist)

Could humans have correctly viewed images in six separate sloshing, bubbling propellant tanks and correctly decided in less than 8 seconds to terminate thrust or not? I doubt it. In fact I doubt that such images would even be clear. It would be extremely difficult to illuminate the tank interiors well enough and keep the camera lenses clean enough to see the other end of a long tank clearly enough to correctly judge by eye that they were dry. Probably the tanks would be filled with fog from the LOX and LH2 that would obscure all images anyway. I know that it is pretty hard to just look at a bowl of LN2 and know when it is dry. And that is a small bowel in a well lit room. Look at some of the rocketcam images at http://www.eclipticenterprises.com/gall ... ketcam.php and notice the variable quality, including occasional loss of signal. No, these decisions need to be made by an onboard computer.

http://www.space.com/missionlaunches/sf ... 50118.html says So the sensors are not in the tank but in the pipe to the Engine. If this pipe runs dry for any reason, even if the tank is full, the engine will be starved. So the people would have to decide that there was fuel in the tank, that it SHOULD be flowing in the pipe, that it is EXPECTED to resume flowing soon, and wait JUST long enough before manually issuing a shutdown command. In 6 different pipes connected to 6 different tanks. All in less than 8 seconds. This is just not humanly possible! Only a computer can make decisions so fast.

Currently I am not considering human control of any processes yet - I am only considering the possibility that humans can check if the data provided to the computers are correct if compared to the reality. control will be considered later.

Obviously there are at least three different places data, video or sound can be got from:

1. engine,
2. pipe and
3. tank

so the following checks are possible at least:

1. Are the data indicating what the real time video shows for

I. engine,
II. pipe and
III. tank

?

2. Does fit into the expected image what the three videos show?

For example the video of the tank may show that sufficient propellant is left while the video of the engines shows that it gets no propellant.

The computer can't use videos currently as far as I am informed - it isn't aböle to interpret them.

Fog in the tank would indicate that there really still is propellant in the tank. It is possible to isolate the webcam from the inner of the tank. But more importantyl at least one webcam should be near the point where the pipe is connected to the tank. And perhaps a way could be found to avoid fog. It is possible too to use cameras watching a wavelegnth where the propellant is brightest.

Regarding the amounts of places, engines etc. to be watched and checked etc. I didn't say that one human only would watch all the videos or data.

But I am talking about a dialog for checking data against real time videos - such a dialog may be controlled by the computers partially as is the usual case in computer-human dialogs: There may be at least one computer which has a schedule when a human should do a check and which can decide which human has to do it. Or the computer may switch the dialog between several places. An additional possibility is that there can be one larger image and several small images.

...

Please note - currently I am considering watching and checking only - but no human intervention: That question has to be left for later.



Dipl.-Volkswirt (bdvb) Augustin (Political Economist)

They already do that. That is why there is a huge army of flight controllers at each launch. What you are really proposing is adding different sensors (cameras for example) that the people would refer to while checking the computer's decisions. And they already do that too, (see rocketcam above). The problem is that things just happen too fast in a rocket launch for a person to actually do anything about it until it is too late. And the whole idea of adding manual human input instead of adding more automation works against the lower cost launch capability we all want!

Peter,

you quoted me saying that I don't have in mind human input:

So don't argue about or against that. You don't know no images of mine about such things currently yet and I didn't say nothing about them an so there is not a single reason to assume something about them.

Fact is that in the case of the Delta 4 Heavy flight for at least one it was unknown what had gone wrong with it - it lasted one day at least until it had been found out that the sensors transmitted wrong data to the computer because of bubbles.

Fact also is that in the case of the Volna-rocket carrying Cosmos 1 nothing is known about what was wrong.

Fact third is that the burn time of the Volna-rocket was 83 seconds and not a few seconds only.

Even the Delta 4 Heavy had a burn time longer than a few seconds only.

A fourth fact is that the emploees of the company I am employed by find more than two seconds reply-time of the computer too long.

So I don't trust currently that the Delta 4 Heavy flight has been controlled by comparing sensor data to real time video of tank, pipe and engine.

The dialog I have in mind shows the sensor data direct in or besides the video - video and sensor data are integrated into one and the same window.

To the dialog additional assisting data can be added - a reference-video which shows how it looks if all is going well and reference-data transmitted by the sensors if all is going well.

The humans doing the checks and comparisons can and should be trained.

Attention can be triggered if the dialog uses an acustic alarm which is focussed on one special person. Each alarm can be focussed on another person and the time of the alarm can be choosen randomly by the computer - but the time shouldn't be shorter than 2 seconds and no longer than 10 seconds. This would allow for 8 to 41 checks and comparisons within 83 seconds - at least.

To clarify:

1. I am talking of at least two different computers currently - the computer aboard the vehicle/rocket controlling it and the computer at the ground station doing the dialog, triggering attention and so on.

2. I am NOT talking about human input to the computer aboard the vehicle.

3. Of course the videos and data should be stored on harddisk, CD or tape.

4. I am currently talking about the dialog only here and its design. Other things will be added to the consideration(s) later.



Dipl.-Volkswirt (bdvb) Augustin (Political Economist)

Ekkehard,

What you're proposing seems to be trying to fix a problem with the sensors by creating a backup for them, whereas I think the real issue should be improving the existing sensors to give better and more reliable feedback.

It is likely to be possible to have such video streaming (I don't know the specifics of bandwith available and how much power that would consume, but I know that SS1 had at least one camera on it), but I don't believe it is the right solution to the problem.

And I agree with Peter that human reaction time makes it not the best solution all the more.

Regards...Dan

Such Dialogs are no argument against improvements of sensors or sensor systems - the improvements should be done really. But malfunctions of sensors are not the only source of the computer working with errored data: the errors may be caused by undetected software-bugs errors and failures of hardware interfaces and much more.

You have recognized correct the current focus and purpose - except that I don't have in mind any human reaction currently - such reactions and the time required shouldn't be discussed yet because the discussion on this dialog stil is missing something.

Currently the only purpose of the dialog would be that a significant amount of time would be saved - so that the public could be informed earlier what had gone wrong.



Dipl.-Volkswirt (bdvb) Augustin (Political Economist)

As long as the true cause of the failure is finally known, I don't see the value in learning it 1 minute after loosing the vehicle instead of 1 week after. A week later is soon enough to fix the problem in the next flight, but 1 minute, or even 1 second later is not soon enough to save THIS flight. They know the Delta had fuel left in the tank, even without additional sensors. Adding more sensors, people, training, computers, displays, and so on, will not improve reliability or safety, but it will drive costs up. There are too many things like that now in a rocket flight. We need to reduce such complexity and intense human attention to the inner workings of the vehicles and just get a space craft that works and one or two pilots just drive it.

The public in general has problems with waiting a week. The longer the public doesn't know the cause of the failure the more their doubts are growing that the constructors can create a failure-free vehicle.

For this reason most companies in case of a failure struggle to show that they know the cause of the failure. It's simply PR up to now.

But I realy will extend it beyong PR and the like - this is the Technology section and the dialog retrictid to checks and comparisons is the beginning only. I mentioned the faster availability of infromations about reasons only to point to a use the dialog would provide. (Not to forget that decisions can be done earlier and faster if these information is available earlier).

Alright - to add something to the dialog: The computer showing the real time video as well as the data can compare the data got from the onboard-computer to the expected data and output a special alert in case of deviations. This special alert would call the humans to the dialog and they could check by the video(s) if the alert is justified or if there is a malfunction anywhere between the sensors, tanks, pipes, engines on the one side and the onboard-computer on the other side.



Dipl.-Volkswirt (bdvb) Augustin (Political Economist)

They really do already do that. There are countless sensors feeding countless computers, in the rocket and on the ground, numerous cameras looking at the vehicle from many angles, including onboard cameras now with rocketcam, countless automatic alerts are enabled, and hundreds of highly trained people are looking at all this data and making split second decisions. And you want to add MORE?

It seems that there are still no cams looking into the tank(s), Pipe(s) and engine(s) - so, yes I wnat more. (Should be clear since I initiated this thread Peter.

I know all what you say but it seems that it doesn't providevisual informations that could be got by today's technology.

Cams can be of soize in millimeter-range today - they are already in use by doctors. Doctors use them to look into the humans body at locations where no normal-sized cam could be moved to. The normal-sized cams would require operation, section or how it is called in English if a doctor opens the body of a living human.

These miniturized millimeter-scale or in future micro-scale cams don't weigh much and could be used abourd rockets and vehicles too to look where currently no cam looks to.

Yes I do want to add more and it is this (Why do you ask for it now but not in the beginning by your first post in this thread?)

The purpose is to provide real time video of something no v+real time video is provided of yet.

This would enable something beyond checks and comparisons which is NOT the human control of the engine etc. What this is I will say in my next post.



Dipl.-Volkswirt (bdvb) Augustin (Political Economist)

In terms of comparing in realtime images of what is correct vs what is not correct, this would probably introduce more uncertainty into the issue. Image processing has progressed a lot in recent years, but still requires a lot of processor power (read v high batter consumption). And generally there is not a high enough certainty in the comparisons - you're looking at 70% to 80% if you're doing well. This is not high enough for a safety critical system like a rocket. In attempting to add more safety by this route, you'd end up making it less safe.

I also agree with Peter that complexity is often the source of many problems. In engineering, the old saying is that you spend 20% of the time on 80% of the work, and 80% of the time on 20% of the work. In increasing complexity, you will end up spending more and more time (and money) on that last annoying 20%. You have to prioritise, and it doesn't seem to me that you've given a strong enough reason to validate spending the resources on such a system.

Please don't ignore that I said

So I didn't say that humans should control the engines, command them and so on - I didn't say that and I don't have that in mind.

Please always recall the following quote of one of my own posts:

and keep alwas in mind the title of this thread too: Dialogs for computer-human-teamwork

(I would be interested in some clarifications why human input into engines, human reaction to processes in drives and so on is considered althought I explicitly say that I don't have in ind something like that - but please don't clarify that here but in the Off-Topic section or by PM)

The mini- and micro-cams need to be connected to the antenny mainly - may be that the computer has to be the interface but the cams don't need to be connected to the engine - their purpose isn't to control anything but only to provide realtime video information to human(s)

If the dialog for checking and comparing is installed and established as well as the cams - what then is there information about? About the question if what the computer knows and reacts to is what is reality.

And what does the title say about the topic of this thread? Dialogs for computer-human-teamwork

The result of the checking and comparing could be provided to the computer aboard the vehcile or rocket. It would be sufficient if the human transmits an information to the computer only if the dta used by the computer don't fit inmto the reality shown by the webcam.

If for example the data used by the computer say that the tank is empty but the cam shows that there still is a significant amount of propellant then it may be reasonable if the human(s) inform the computer that there still is a significant amount of propellant.

This way an errored shutdown of the engine(s) could be avoided perhaps - the failure of the Delta 4 heavy may have been a little bit smaller perhaps. But all this will be considered more detailed here.

But in this post the essential thought only is that the computer could be informed of soemthing he has don't know - he can be informed about an error included in his input data.

What the computer does with this information is determined by his software which is not on-topic here - except the possibility that it may have interfaaces to dialogs.

Please note - the human's only informed the computer about something - but they don't command him what he has to do, they don't recat to no process: The humans only react to whatbthey see by video and the data they read assembled in one and the same dialog.

And currently I consider the case that they will react only if data are wrong and may cause a failure.

Its teamwork between humans and the computer and the computer alone has control of the engines - the computer alone commands the engines.

It's a talk to the onboard-computer only now - adialog between that computer and humans: No engine is involved in this.



Dipl.-Volkswirt (bdvb) Augustin (Political Economist)