Open Stack

Great feedback, thanks!

A precise unambiguous time is the primary requirement for expiration dates of secrets in barbican (so no timezone info need be preserved). Hence UTC-only seems like the most fitting approach to specifying such dates then. As such then, requiring UTC-only in the API sounds reasonable given this requirement.

As for implementation, there might be a need for UTC-only parsing additions to oslo, maybe even a fixed strftime sort of approach to really tighten things up. :)

Thanks,
John

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

________________________________________
From: John Dennis [jdennis at redhat.com]
Sent: Wednesday, June 05, 2013 10:24 AM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [barbican] Date/time/timezone parsing

On 06/05/2013 08:36 AM, Jarret Raim wrote:
> On 6/4/13 9:13 PM, "John Dennis" <jdennis at redhat.com> wrote:
>
>> On 06/04/2013 07:01 PM, John Wood wrote:
>>> Simo, we were planning to normalize times into UTC prior to putting
>>> into datastore, but didn't know if it would be too stringent to make
>>> clients also conform to UTC to use the API. Using UTC on both sides of
>>> the API does seem safer and more robust overall though, so we could
>>> enforce this in our code base for sure.
>>>
>>> Would anyone out there object to a UTC-only mode of operation for
>>> barbican?
>> Time values should always be in UTC. The only time (no pun intended) a
>> time value should be in local time is when it is displayed to the user
>> or accepted as user input, after which it should immediately be
>> converted to UTC. Following the rule that time values are always UTC
>> will prevent any number of nasty problems that can easily be avoided.
>
> The API will store all date times as UTC. However, when a customer
> specifies a timezone offset in a message, we have two options. Either we
> accept the message, modify the UTC time to correctly represent the
> requested date time (e.g. Apply the offset) or reject the message as
> malformed.
>
> The current iso8601 implementation allows us to do neither. In some cases
> it incorrectly parses the timezone offset (or ignores it) and does not
> throw an error. I'm fine with rejecting a message with an offset if that's
> the way that the rest of the APIs work. Is there a way to do that with the
> current olso / iso8601 implementation? I guess we could roll our own, but
> that seems like something property belonging to the parsing library.
>
>
Going back to the examples in the original post it appears the parser is
broken and needs to be fixed. Here are my suggestions based on having
worked a fair bit with date/time values.

Follow the rules in RFC 3339.

The wikipedia article on ISO 8601 gives a very lucid (i.e. not
rfc-speak) of how 8601 works, from that you can see the parser is broken
with respect to "last field, highest precision" parsing. Fix the parser.

Demand that any message containing a timestamp which does not strictly
follow the rules be rejected. Permitting common case exceptions or other
formats has lead to many headaches best avoided. Make clients play by
the rules.

Never accept a timestamp without a timezone specifier, either "Z" for
UTC or a numeric offset.

Provide guidance that timestamps should be in UTC.  My personal
preference is UTC but in some scenarios knowing the offset can be
useful, but this usually only arises in select situations such as in
scheduling where an awareness of local time can provide extra
information useful for allocating resources. But there are so many
problems with the interpretation of UTC offsets that there be dragons
there, beware. The best recommendation is to immediately convert to UTC
but preserve the offset as extra information if you believe it might be
useful, but don't use the stored offset for anything authoritative,
consider as only scheduling guidance if needed.

I don't know if the Python module you're working with is based on
Python's native datetime or not but I've used datetime a fair amount and
(at least in Python 2) there is a nasty problem with respect to
timezones. It appears as if datetime was originally written without
timezone support and timezone was later grafted on. This lead to "tz
naive" and "tz aware" datetime objects. This was very unfortunate
because a timestamp with tz info isn't much use as a timestamp because
it's ambiguous. It's further compounded by the fact you get "tz naive"
datetime objects by default, which of course is what most people do. So
by default you get meaningless timestamps :-( It's kind of like having a
scalar value without pairing it with a "units" value. Because Python's
datetime objects are so ambiguous (either by default or by incorrect
processing of the extra tz info) the clearest, simplest, and most robust
recommendation is require that every datetime (or time) object be in
UTC. Hopefully it will be obvious a "tz naive" datetime object can't
distinguish between a value in UTC or one that has an implicit offset,
hence the rule "UTC always".

Of course there is problem of comparing datetime values. If datetime
objects always carried tz offset then using non-UTC values would be fine
in most cases because the values would be normalized before comparison.
But without the extra tzinfo data you can't normalize. Python's datetime
library will throw an exception if you try to compare a naive and aware
object because normalization can't be performed. But it will happily
compare two naive objects and unless those objects happen to share the
same tz offset you're screwed. You're really left with two choices 1)
always use "tz aware objects", but this is difficult because you don't
get it by default and it's non-trivial, or 2) always use UTC.

Hope this clarifies things a little.

The biggest lesson I've learned from systems with problems (and there
are many) is initial weak specification of date/time values and/or
permitting exceptions, in short order you'll have a mess. The other
thing to remember is UTC offsets are very tricky and are only meaningful
for an exact time and location (hence the Olson Database). Try as much
as possible to avoid offsets, if present process them immediately
(stored offsets are a huge problem) and treat a supplied offset as
informational extra data or completely discard it if local scheduling is
not involved.

John

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev