ExpressionEngine CMS
Open, Free, Amazing

Thread

This is an archived forum and the content is probably no longer relevant, but is provided here for posterity.

The active forums are here.

UTC epoch value is not what's stored in database

June 25, 2008 12:46am

Subscribe [4]
  • #1 / Jun 25, 2008 12:46am

    Dan Halbert

    93 posts

    This is EE version 1.6.3. My EE setup is pretty vanilla. The Server Time Zone is Eastern time, and the Server Offset is blank.

    In several places it is stated that EE stores UTC times in the database. But this does not seem to be what I am seeing.

    On my server, I did this to see the current epoch value:

    $ date +%s
    1214364366

    which is:
    Wed, 25 Jun 2008 03:26:06 GMT
    Tue June 24, 2008 11:26:06 PM EDT
    That is correct UTC. It was indeed 11:26PM here (my server happens to be in my time zone).


    Then I immediately created a weblog entry, and checked its timestamp in the entry_date column in exp_weblog_titles. There I see the timestamp:

    1214378760

    That is 14394 seconds later, not quite 4 hours, which is:
    Wed, 25 Jun 2008 07:26:00 GMT
    Wed, June 25, 2008 3:26:00 AM EDT
    (it’s earlier in the same minute because it dropped the seconds, I guess)

    So why is EE storing that weird adjusted non-UTC epoch time? All the KB articles, wiki articles, and forum posts say it’s UTC. Why is EE bothering to adjust it?

    Scratching my head,
    Dan

    (This all started because I modified the CSVgrab plugin to use strtotime() to convert a date and time to an entry. I was getting times that were hours off, and was pretty puzzled.)

  • #2 / Jun 25, 2008 12:33pm

    Robin Sowell

    13255 posts

    I can’t replicate- but did it differently.  Added an entry- turned php parsing on, parsed on input:

    <?php global $LOC; ?>
    
    {exp:weblog:entries limit="2" weblog="reading"}
    {title} - {entry_date} - <?php echo $LOC->set_human_time('{entry_date}', FALSE); ?> - {gmt_entry_date format="%m, %d %Y  - %g:%i:%s"} - {entry_date format="%m, %d %Y  - %g:%i:%s"}
    
    
    
    {/exp:weblog:entries}

    Output is

    Date check 11:23 eastern as I post - 1214425080 - 2008-06-25 03:18 PM - 06, 25 2008 - 3:18:00 - 06, 25 2008 - 10:18:00

    All of which matches up.  Can you walk me through it a little clearer?  Or- may be an issue involving server settings.  Can’t swear on that- but my dates look to be going in UCT.

  • #3 / Jun 25, 2008 1:15pm

    Dan Halbert

    93 posts

    Hi - I am talking about the value that’s actually stored in the exp_weblog_titles MySQL table, in column entry_date. Try adding a weblog entry and then check the value in that column for the new entry.

    You can find out what the UTC epoch value should be and convert it to human-readable form here: http://www.epochconverter.com.

    I can believe that EE transforms to/from UTC going in and out, as you showed above, but it seems like it stores a non-UTC timestamp in the table. That is what I was trying to show. It appears to add a timezone/DST bias, and then undo this when it takes it back out. You won’t see that if you just use EE constructs, as you did above. You have to look inside the table.

    The problem is that various plugins and extensions don’t deal properly with this bias. I’ve actually never seen any other software that stores non-UTC epoch values, assuming that’s what you’re doing. I would have expected you to store a standard UTC epoch and then just use a standard date formatting routine that prints the date/time in the local timezone of the user. The way EE does it, the stored timestamp is not UTC, but is dependent on some setting that could change (assuming my surmise is correct).

  • #4 / Jun 25, 2008 5:36pm

    Ingmar

    29245 posts

    I would like to rule out the possibility that you getting different results on the commandline and via PHP/EE. So, can you please use

    {current_time format="%U"}

    in an EE template, and compare with “date +%s”?

  • #5 / Jun 25, 2008 7:39pm

    Dan Halbert

    93 posts

    Sure. Here goes:

    $ date +%s;date
    1214432753
    Wed Jun 25 18:25:53 EDT 2008

    And then immediately afterward, in a template:

    { current_time format="%U" } : 1214432762

    9 seconds difference.

    A few seconds after that, I updated the entry date in a weblog entry, and looked in the entry_date field of exp_weblog_titles:

    1214447160

    14398 seconds difference.

    The date in the Edit Weblog Entries page for that entry (which is as expected):

    06/25/08 06:26 pm

    Server Time Zone is Eastern Time, no Server Offset

    I think EE is operating as designed (set_localized_offset() et al in core.localize.php); I’m just surprised you did it that way.

    What worries me is that is seems that the timestamps stored in exp_weblog_titles are dependent on the timezone setting, instead of being UTC and therefore timezone-invariant. So if the data in that table gets migrated to a server in another timezone, the timestamps will all be off. Also, I have to adjust any timestamps I put in the table directly (e.g. with CSVgrab). Am I misunderstanding?

  • #6 / Jun 26, 2008 2:33pm

    Lisa Wess

    20502 posts

    Hi, Dan,

    Can you let me know what your own account localization settings are? You can find them in the CP, in My Account -> Localization.

    Thank you.

  • #7 / Jun 26, 2008 3:12pm

    Dan Halbert

    93 posts

    Hi - this is the initial admin account. There are no other accounts: I’m still building the site.

    Select Time Zone: Eastern Time
    Daylight Saving Time: box is checked
    Time Formatting: United States  
    Select Language: English

  • #8 / Jun 27, 2008 12:56am

    Dan Halbert

    93 posts

    Sorry, I am still chewing on this bone. I looked at core.localize.php, and particularly, this function:

    // Takes a Unix timestamp as input and returns it as GMT
        
        function set_gmt($now = '')
        {    
            if ($now == '')
            {
                $now = time(); 
            }
                
            $time =  mktime( gmdate("H", $now),
                             gmdate("i", $now),
                             gmdate("s", $now),
                             gmdate("m", $now),
                             gmdate("d", $now),
                             gmdate("Y", $now),
                             -1    // this must be explicitly set or some FreeBSD servers behave erratically
                           );   
    
            // mktime() has a bug that causes it to fail during the DST "spring forward gap"
            // when clocks are offset an hour forward (around April 4).  Instead of returning a valid
            // timestamp, it returns -1.  Basically, mktime() gets caught in purgatory, not 
            // sure if DST is active or not.  As a work-around for this we'll test for "-1",
            // and if present, return the current time.  This is not a great solution, as this time
            // may not be what the user intended, but it's preferable than storing -1 as the timestamp, 
            // which correlates to: 1969-12-31 16:00:00. 
    
            if ($time == -1)
            {
                return $this->set_gmt();
            }
            else
            {
                return $time;
            }
        }


    set_gmt() claims to take a Unix timestamp and convert it to GMT. However, it’s really encoding a GMT time by converting it into the local timezone.

    If you call set_gmt() with no arguments, it starts with a call to time(). time() returns a Unix epoch timestamp, the number of seconds since Jan 1, 1970 UTC. The timestamp is already GMT, not local server time. If you called time() on a bunch of Unix servers anywhere in the world at the exact same time, they would all return the same timestamp, assuming they are set to the correct time. Their local timezone doesn’t matter.

    set_gmt() then disassembles the timestamp into its parts with the gmdate() calls, and creates a new timestamp with mktime(). But mktime() assumes that the inputs are in local time. So the function ends up adding the difference between local time and GMT. I tried this in a little test program and saw that if I use the “now” time, the output time is 4 hours ahead (I am in EDT now). Here’s the output of my program:

    1214535912 Thu, 26 Jun 2008 23:05:12 -0400      # time(), date('r', time())
    1214550312 Fri, 27 Jun 2008 03:05:12 -0400  # set_gmt(), date('r', set_gmt())

    So set_gmt() is encoding the GMT time by converting it to a time in the local timezone. So if it’s 11pm here and 3am in Greenwich, this function returns a time that says it’s 3am here.

    Since all the functions in core.localize.php use this same scheme, they are internally consistent, even though they use this conversion, and EE presents times that are consistent. The problem comes when I try to relate these special EE timestamps to the way epoch timestamps are usually used, which is always GMT by default. Then all the timestamps in the EE database are ahead of what they might be by the difference between the current time zone and GMT.

    And, if the installation moves timezones (by moving the server or the data), the times will all be off, forever, because when they were put in the database they were offset from GMT based on the original timezone. So it seems to me it would be better to store only real GMT times in the database.

    I am sorry to be so pedantic. EE has been using timestamps this way for years, I suppose. Changing it is a big deal. Now that I understand how they are different than what I expected, I can adjust my own code accordingly. But both the documentation and the code say “time is stored in GMT”: I interpreted that in quite a different way, and maybe others have too.

  • #9 / Jun 27, 2008 1:06am

    Lisa Wess

    20502 posts

    Sorry, Dan.  I’ve been following up on this with the crew.  In regards to this statement:

    What worries me is that is seems that the timestamps stored in exp_weblog_titles are dependent on the timezone setting, instead of being UTC and therefore timezone-invariant. So if the data in that table gets migrated to a server in another timezone, the timestamps will all be off.

    This in particular is something that will be addressed in 2.0. 

    At this point I’d like to confirm that your dates are correct and the display is outputting correctly.  Are the entry dates reflected appropriate on your public site?

  • #10 / Jun 27, 2008 1:16am

    Dan Halbert

    93 posts

    This in particular is something that will be addressed in 2.0.

    Wow, great! And, yes, there is no problem with my current date/time displays. I only discovered the issue when I tried to go through the back door by modifying CSVGrab to import some data from an existing non-EE calendar system.

    Dan

  • #11 / Jun 27, 2008 2:24am

    Lisa Wess

    20502 posts

    Hi, Dan,

    I’m glad to hear that everything is working.  If you have a feature request regarding how time and date stamps are handled, then I would ask that you post that in the feature request forum.  But for now, as everything is working correctly and as expected, are we ready to close this one out?

  • #12 / Jun 27, 2008 8:34am

    Dan Halbert

    93 posts

    Yes, fine to close. Thanks!

  • #13 / Jun 27, 2008 8:36am

    Sue Crocker

    26054 posts

    Closing the thread..

.(JavaScript must be enabled to view this email address)

ExpressionEngine News!

#eecms, #events, #releases