CHAPTER 4

The Representation of Time

4.1 Time-stamps

In the previous section (3.5) we considered an example which included the statement 'move it back to its original position'. To deal with such a statement, and with its use of the word 'originaf, we require our NL system to have some sense of time - of the chronological sequence of events. Fortunately the means to deal with this is to hand, although a full treatment is fraught with difficulties.

If we printed the internal text and laid it out for inspection, we would see a sequence of lines of code, each of which was created at a different moment in time. If our computer had a system clock we could have appended a 'time-stamp' to each line of code to identify theinstant at which it was created. And herein lies a clue as to how we might deal with the representation of time.

We should note that there are at least two kinds of clock. Some clocks begin counting from zero when the computer is switched on. We might describe that as 'relative time', since it is relative to the start of each session. Others, which have a built-in battery, record 'Greenwich Mean Time' or some similar standard time, and so we might describe that as 'standard time'. There is of course no such thing as truly 'absolute time' since all time values are relative to some arbitrary standard, but there is one absolute point in time which corresponds to the time at this very instant, which we shall call 'now'.

There is also one invariant property of time. Given two times T 1 and T2 such that these are not equal, if Tl < T2 in some measure of time, the same relationship will hold no matter what measure of time we use. There are some peculiar circumstances which arise in high energy particle physics but these lie outside the scope of our discussion. Another point to note is that if a line of code is such that it refers to some moment in time, then we actually have two times to consider. To explain this it is best to leave the micro-graphics world for a moment and consider a database of birth records. For each record there is the time (date of birth) to which it refers, and there is the time at which the record was inserted into the database. These are not necessarily the same and usually will not be. When we wish to make this distinction, the time at which the record was created will be called the 'physical time'.

With these ideas established let us now consider the sequence of events laid out for inspection in our listing of the internal text.

4.2 Origins

Returning now to our micro-graphics world, if we inspect a listing of the internal text we would be able to recognise that some of the lines of code (not necessarily adjacent) might be considered as forming a group which defmes an entity. Three lines of code might specify a triangle, for example. Another entity (a point, for example) might be defined by a single line of code. At any instant in the chronological sequence of coded lines, an entity will have a particular set of parameter values (e.g. a colour, or values for its coordinates). These values will be provided by our value table (as described in section 3.2). The implication is that the rows of the value table will need to be time-stamped. The values will remain fIXed until they are changed by some other line of code.

Consider now the example shown below. In this example most ofthe lines of code are of no interest and we will represent them by dotted lines ( ....... ).




We have denoted the time-stamps by to, tl, t2, t3,... The lines timed at t2, t4 and t6 define a triangle. The question which now arises is this. At what time did the triangle pqr come into existence? We submit that the correct answer is 't6', because that is the time-stamp value of the line r-p, and until that line was drawn the triangle did not exist (although two sides of the triangle did exist before that time). From t6 until tlO the parameter values of the triangle pqr remained unchanged. At tlO one of the lines (pq) was redrawn in a different colour (O=white, I =red). The original incarnation of the triangle therefore had an existence which extended over the period (t6-tlO). An argument could be developed along the lines that an object need not have an origin which is an identifiable moment in time, but it is sufficient to note here that the issue is not entirely clear-cut. If that seems rather vague the reader should be warned ­worse is to come.

4.3 Object Histories

By examining the internal text like this, we could identify the history of an object or entity, leading from its origin through a series of states, each with a particular set of parameter values and its own time-period. But to talk about this we need some terminology.

A 'state' is a condition which lasts for a period of time and during which the properties of a specific entity or collection of entities remain unchanged. A state therefore has two logical time-stamps corresponding to its starting time and its ending time. However the ending time is not shown: it must be. discovered by examining the other states and finding the time-stamp of the next state which is different.

An 'event' occurs when there is some change of the properties of an entity or set of entities. An event is therefore a change of state.

The time-period associated with each state indicates its place in the chronological sequence. Other statements and entities in the internal text could be labelled in the same way to produce multiple traces showing the entire history of events leading from the start of the dialogue to the logoff message.

We can recognise that an entity may have persisted over an extended period of time, through several incarnations, and we could define a 'life-time' or 'object history' for the entity, starting at the origin when it was first defined and terminating when it was deleted by redrawing it in background colour. But if the identification of the start of an object history was difficult, the identification of the termination of an object history is doubly so. Redrawing in background colour occurs many times during a move process, and so a difficult problem now arises concerning the point at which we can definitely consider the entity as having been 'killed off'. We might try, for example, to define a maximum period of extinction. Less than this period we may consider to be 'momentary' and a longer period will be considered to be a definite 'death'. When we consider how human beings perceive motion in a rapid sequence of still photographs (e.g. a cine film), we see that our notion of the 'momentary' extinction, which can be ignored, has some basis in fact. The visual mechanism appears to retain the image of an object and fuse it with another which appears within a short period oftime. There is, however, another form ofretention which lasts much longer, and that is the focus of attention.

If an entity is extinguished from the screen we can wait for quite a long period of time and then make the request 'redraw the triangle.' This is possible because the memory of the triangle remains. It may have disappeared from the screen, but it has not disappeared from that 'inner eye' of the mind. The focus of attention remains on the object recently extinguished. This is only the case if nothing of any significance has happened in the mean time which could have changed the focus of attention.

Now - if the triangle is redrawn, has the life-time of the original triangle been extended or are we talking about the start of a new life-time for a new triangle?

The problem is actually 'undecidable'. The point we wish to make is that the life-time of an entity depends upon the perceptions of the viewer (or the perceiver) and their focus of attention. It is not a simple matter of time periods and physical extinctions. The question above can be answered correctly in either way, provided one assumes a suitable context.



4.4 Before and After

Now suppose that our user requests the system to draw a red triangle, move that triangle about and then change its colour to green, move it about again and then change its colour once more to yellow. The sequence is: red( to)-move(t 1 )-green(t2)-move(t3)- yellow(t4).

Here the triangle was red from to to tl, it was moved during the period tl to t2, and so on. The 'present' time is represented by the system clock and the most recent time-stamp is t4. We shall represent the current value of the system clock by 'now'. Our user might then make the statement 'Before making the triangle green move it upwards 10 units.' The user is inserting an additional command retrospectively (physical time = 'now', logical time = before(t2» and we have introduced by implication the idea of the 'action replay', that is, that the internal text can be re-run to produce the graphic images once more at a later time. This will require the regeneration of the text as for a 'macro-expansion' as we described in section 3.3.

The word 'before' refers not to a time period before the present time ('now'), but to a time before t2, which was the time at which the triangle was made green (in each of the incarnations of that internal text). The time-stamp t2 is relative to the other time-stamps in the sequence. In terms of standard time it could be in the actual past (before 'now') or it could be in the actual future (during a future action replay). The time-stamp 't2' therefore has multiple occurrences (in each incarnation of an action). Each time-stamp within an incarnation therefore has a 'standard' component (different for all versions of t2) and a 'relative' component (the same for all versions of t2).

Notice that the focus of attention has been switched to t2, which now becomes a kind of 'pseudo-present' time. That is, 'before' and 'after' refer to before(t2) and after(t2). The pseudo-present status oft2 can be seen in the fact that the user can change the tense of the verbs to present tense while still referring to t2. For example 'As it is becoming green it should flash rapidly.' Note that 'As it is becoming ...' is in the present tense even although the user is referring to t2, which in its original incarnation was in the past (relative to 'now').

If there are two green triangles on the screen the user may refer to '...the green triangle which will become yellow...'. Here we see even the future tense '.. will become..' can be used to refer to an event which is in the standard past, but is in the future relative to the time associated with the focus of attention (t2).

To interpret the statement 'Before making the triangle red move it upwards 10 units' the NL-system should:

(a) create a representational pattern corresponding to the making of the triangle green (code-segment).

(b) initiate a search back through the internal text to achieve a match with the pattern. (Note that the pattern matching technique will be needed - see Chapter 8).

(c) copy the relevant segment of internal text to create a new incarnation of that segment

(d) insert a new line (or lines) of code which moves the triangle upwards 10 units.

(e) re-run text at absolute time 'now'.

By inserting the lines of code for a change of position we leave behind in the internal text a reference object for such statements as 'and after making the movement make the triangle flash'.

The new incarnation is essentially timeless in standard time until it is re-run on the screen, but the (relative) relationships between to, tl, t2, ... etc remain constant (to occurs before tl which occurs before t2, and so on). It therefore seems that although we need to represent both standard and relative time components, there is no necessity for more than one parameter. We can assign identifying labels to the time-stamps (e.g.: to, tl, t2, ...). We can then declare certain properties which we wish these to have (e.g.: to < tl < t2 < ...). We can also relate any of these to a standard time scale (e.g.: t2 = now). Once t2 is known to be equal to 'now' we can make certain deductions about the other time-stamp parameters (e.g.: tl < now). This seems a flexible and economical way to represent time. When a segment of code is copied to produce a new incarnation of it, the relative time relationships must be copied as well. The whole sequence of events can be located in standard time later by relating the standard value 'now' to one of the time-stamp parameters. This relationship to 'now' is a property of the incarnation, not of the individual states within that incarnation. If a sequence of events takes place in three steps at to, tl and t2 we can place the whole action in the past by declaring 't2 < now'. The whole action can be declared to be in the future by the statement 'to> now' and the action can be declared to be actually happening in the present by the statement 'tl = now'.

The idea of multiple incarnations of a sequence of events, each of which can be separately located in standard time, has its counterpart in storytelling. We might say 'Hamlet will kill the king' (future tense) although the actions have been portrayed innumerable times in the past (in previous incarnations or performances).

4.5 Tense

These ideas fit rather nicely with the way time is expressed through the tense of verbs. The present continuous (e.g.: 'it is moving') is equivalent to 't 1 =now'. Past tense (e.g.: 'it moved') is equivalent to 't2 < now', and future tense (e.g.: 'it will move') is equivalent to 'to> now'.

The more complex tenses are expressed by more complex verb structures (e.g.: 'it has been moving'). In the example above we used t I to indicate the time at which the action is in 'mid-process'. The tense of the verb above indicates that that point in the action is in the past 'tl < now'.

To deal with 'it will have been moving' we must introduce another time-stamp parameter (t3). The tense of the verb indicates that t3 is in the future ('t3 > now') and that the state of mid-action (tl) occurred before that ('tl < t3'). From this we cannot deduce the relationship between tl and absolute 'now'. All we know is that some time in the future the action will have been in the past (relative to the new future 'now').

Finally the infmite tense (e.g.: 'it moves') corresponds (except in special (real­time) storytelling techniques) to the situation in which there is no relationship to absolute 'now', It is essentially timeless.




4.6 Simultaneous Actions

The use of the words 'while' and 'during' in the input text suggests the idea of parallel execution in the relative .time of the graphics display. Present-day computers have some difficulty with parallel execution, but future machines may not have this difficulty. In the mean time it is often possible to achieve pseudo­parallelism with a swapping mechanism which executes two or more sequences of instructions by swapping from one to the other, one instruction at a time.

A possible solution might therefore be the introduction of a swap macro­generator, in the same way as we used the move macro-generator.

We need not concern ourselves with the actual mechanism by which parallelism is to be achieved, but we note that the internal representation must be able to assign identical relative time-stamps to segments of the internal text. It follows that we cannot rely upon the linear sequence of lines of code in the internal text to indicate relative time. There must be some explicit representation of the relative time component of each time-stamp (as well as the standard component).

The suggestion is, therefore, that each line of code in the expanded internal text is provided with a time-stamp parameter (to,tl,t2,t3,...) and that in addition we have a database of 'facts' about the values of these parameters. These facts may be global or local (being tagged on to some incarnation). The chronological sequence of the lines of code can be discovered only by reference to this database.

4.7 Summary

In Chapter 4 we have sought to make the following points.

(a) The chronological sequence of events can be represented as a series of discrete states each with a time-stamp, together with a database of facts indicating the relative positions of each time-stamp parameter.

(b) Objects have object-histories but the starting and ending points of such histories are defined ambiguously.

(c) Time-stamps have absolute, standard and relative components. These can be represented by arbitrary time-stamp labels to which are attached relationship information. When the relationship is relative to 'now' the time-stamp has been located relative to the present time. (d) When the focus of attention is directed at a particular event or state the time-stamp (relative component) acquires a pseudo-present status.

We shall return to these issues when we deal with verbs in Chapter 6. First we must digress (in Chapter 5) to study how the important verb endings and structures (which indicate tense) can be recognised.