ExpressionEngine CMS
Open, Free, Amazing

Thread

This is an archived forum and the content is probably no longer relevant, but is provided here for posterity.

The active forums are here.

DataMapper 1.6.0

September 05, 2008 12:32pm

Subscribe [115]
  • #316 / Dec 04, 2008 4:52am

    stensi

    109 posts

    Neat 😊  I hope it wasn’t too much of a hassle changing it all 😊

    How come you store the timezone?  PostGre must handle DateTime fields differently since MySQL doesn’t even let you store the timezone in a DateTime field, lol.  It only stores it in the exact format DM generates (‘Y-m-d H:i:s’).  I added the “O” to double check and yep, it was ignored by MySQL.

    If I was allowing users to choose their own timezone, I’d store the timestamps in the default DM way (local_time = FALSE) and store the user chosen timezone in a separate field in their user record. That way, I can get timestamps from any record (user or other type) and then apply the users timezone to them upon displaying, so dates are always localised to the specific user.

    Alternatively, you can set local_time = TRUE in the config, then it goes by the local time setting of the server and so the timezone is automatically applied when storing timestamps.  Obviously, you wont be able to do the above trick then.

    I’m sure you’re already aware of this.  I was mainly posting in case anyone else might wonder about the possible uses.

    In the next version I’ll be looking at localisation for the label’s so that will hopefully negate the need for those post_ methods.

  • #317 / Dec 04, 2008 5:19am

    OverZealous

    1030 posts

    Well, I wouldn’t be able to switch away from my current localization technique - I include a lot of other information in my files (such as units, singular and plural forms, labels for related fields, and labels for other specialized items).  Most of it is not loaded into the model, but accessed through special functions on my DM extension class.

    Also, I am using those methods in other ways, to manipulate static arrays within a couple of models, and to manipulate the validation array dynamically.  I do this mostly for my SortableModel class, which I think I sent you a copy of, stensi.

    I’ll keep adding them myself, though.

  • #318 / Dec 04, 2008 10:06pm

    stensi

    109 posts

    Version 1.5.3 has been released!

    View the Change Log to see what’s changed.

    I feel stupid that I missed this, but this release is to fix the $parent setting that affects the validation of self referencing relationships only (I’d accidentally left it as an object reference when I should have changed it to an array).

    One other change, that I hope wont cause too many headaches :red: is that the $fields property is back as a normal array of just the field names, making it easier to loop through a properties fields.  Example:

    $u = new User();
    $u->get(1); // Get first record
    
    // Loop through the fields of the user object
    foreach ($u->fields as $field)
    {
        // And show the values
        echo '<strong>' . $field . '</strong>: ' . $u->{$field} . '
    ';
    }

    Enjoy 😊

  • #319 / Dec 04, 2008 10:20pm

    OverZealous

    1030 posts

    I hope you didn’t switch $fields back just for me!  I actually thought that I might try to make use of the database metadata at some point.

    I don’t know if I made it clear, but I did get both CI 1.7 and DM 1.5.2 fully upgraded.  That was one of the biggest framework changes I’ve made.  Went surprisingly smooth.

    Of course, my mail server (on the same box) decided to give me trouble last night, so, that was fun.  I thought my app was running slowly, turns out the mail server decided to eat up 100% of the CPU.  Ain’t life grand!

  • #320 / Dec 04, 2008 10:37pm

    empoleon

    5 posts

    I’ve just started with Datamapper and am uncomfortable with the process of retrieving records when multiple joins are required.  The large number of queries being executed makes me think I’m going about this the wrong way or I should bypass Datamapper for such operations?

    Scenario:
    A Company sends me Products made by another Company and a Job is created.

    Process:
    List all Jobs.  For each Job, list the associated Company and Products.  For each Product, show the Company that manufactured it.

    My controller:

    function show_jobs()
    {
        // Get all Jobs
        $j = new Job();
        $j->get();
    
        // Loop through all Jobs
        foreach ($j->all as $job)
        {
    
            // Retrieve Company associated with current Job
            $job->company->get();
    
            // Get all Products associated with current Job
            $job->product->get();
    
            // Loop through each Product associated with current Job
            foreach ($job->product->all as $product)
            {
    
                // Retrieve Company associated with current Product
                $product->company->get();
            }
        }
    
        // Send data to View
    }

    Does this seem right? Listing all 3 Jobs required 17 queries which makes me nervous considering I’m keen to add more relationships.  But I know I did it wrong- I just know it! 😊

  • #321 / Dec 04, 2008 10:42pm

    stensi

    109 posts

    @OverZealous.com: Well, maybe partly, lol 😊

    Mostly though, I changed $fields because upon further investigation, I found the returned meta data was quite different depending on the type of database being used.  I originally envisioned using it for help with automatically determining a fields data type for display in HTML, but even if the data was returned in a standard way for all databases, I don’t think it would have worked out anyway.  For example, a value might be stored as an integer but the developer might want it displayed as currency, or you could have a unix timestamp stored as an integer, and they wanted it displayed as a date.  Better to have the developer specify the data type in the validation array.

    Besides that, I find the code just looks cleaner when foreach’ing through just the fields, lol 😊

    Ah, the joys of server management.  I recently got a slice at SliceHost.com and am setting it up at the moment.

    @empoleon: Could you post the $has_many and $has_one settings you have in each of your Job, Company and Product models?

  • #322 / Dec 04, 2008 10:59pm

    empoleon

    5 posts

    Here are the relevant tables:
    //————————————————————-
    // NORMAL TABLES
    //————————————————————-
    companies (3 records)
    id
    company_name

    jobs (3 records)
    id
    job_date_created
    job_date_completed

    products (6 records)
    id
    product_name
    product_model
    product_sku

    //————————————————————-
    //  JOINING TABLES
    //————————————————————-
    companies_jobs (3 records)
    id
    company_id
    job_id

    companies_products (6 records)
    id
    company_id
    product_id

    jobs_products (6 records)
    id
    job_id
    product_id

    //————————————————————-
    // MODELS
    //————————————————————-
    class Company
      var $table = ‘companies’;
      var $has_many = array(‘job’, ‘product’);

    class Job
      var $table = ‘jobs’;
      var $has_many = array(‘product’);
      var $has_one = array(‘company’);

    class Product
      var $table = ‘products’;
      var $has_many = array(‘job’); // A ‘Product’ can be on different ‘Jobs’
      var $has_one = array(‘company’);

  • #323 / Dec 04, 2008 11:56pm

    OverZealous

    1030 posts

    @stensi - 1.5.3 ‘upgraded’ easily enough 😉

    Thanks for mentioning slicehost - I had heard about them a while back, but had forgotten.  Amazing prices!  I had gotten a quote from Rackspace - since I need a certain level of control over the server - and they start at almost $400/month.  With the upgrade-ability of slicehost, I could probably start on a much lower plan, and upgrade as I get more paying customers!

    I currently, and for the short term future, am hosting the application out of my home on a business connection.  (Of course, network uptime is sometimes an issue, even if bandwidth, storage, and server management are not.)

  • #324 / Dec 05, 2008 12:21am

    stensi

    109 posts

    @empoleon: The number of queries you get will depend on how many records there are in each table.  I’ll try to explain which code causes which queries:


    This one is run to determine if a corresponding table exists for each of your models:

    SHOW TABLES FROM `your_database_name`


    These are run once per model to get the list of field names:

    SELECT * FROM `jobs` LIMIT 1 
    SELECT * FROM `companies` LIMIT 1 
    SELECT * FROM `products` LIMIT 1


    This one…

    SELECT * FROM (`jobs`)

    ...is executed when you do:

    $j->get();


    This one…

    SELECT `companies`.*
    FROM (`companies`)
    LEFT JOIN `companies_jobs` ON `companies`.`id` = `companies_jobs`.`company_id`
    LEFT JOIN `jobs` ON `jobs`.`id` = `companies_jobs`.`job_id`
    WHERE `jobs`.`id` = 1

    ...is executed when you do:

    $job->company->get();

    You’ll get a similar query each time you loop through the 1st foreach, for each job ID, so you might have multiple ones like this.


    Note that I don’t think your show_jobs() method ever gets into the 2nd foreach because you’re missing a get() on the $job->product.  It should have:

    foreach ($job->product->get()->all as $product)

    This causes this type of query:

    SELECT `products`.*
    FROM (`products`)
    LEFT JOIN `jobs_products` ON `products`.`id` = `jobs_products`.`product_id`
    LEFT JOIN `jobs` ON `jobs`.`id` = `jobs_products`.`job_id`
    WHERE `jobs`.`id` = 1

    Again, you’ll get a similar query each time you loop through the 1st foreach, for each job ID, so you might have multiple ones like this.


    This one…

    SELECT `companies`.*
    FROM (`companies`)
    LEFT JOIN `companies_products` ON `companies`.`id` = `companies_products`.`company_id`
    LEFT JOIN `products` ON `products`.`id` = `companies_products`.`product_id`
    WHERE `products`.`id` = 1

    ...is executed from this code inside your 2nd foreach:

    $product->company->get();


    In one of the coming versions, I plan on including “related_{clause}” methods to help developers fine tune their queries on related objects, to reduce the total number of queries needed.  At the moment, doing the type of thing you have in your show_jobs() method might indeed generate a fair amount of queries, if there are lots of records.  Even so, databases are designed to handle JOIN’s very quickly so will cope fine with this sort of database querying.

    I’d recommend you page your results though, rather than just showing everything, which will save on unnecessary bandwidth usage.  Example:

    $page = 4;
    
    $limit = 10;
    $offset = ($limit * $page) - $limit;
    
    $j = new Job();
    $j->get($limit, $offset);

    That will return a maximum of 10 records, starting at page 4 (so records 40 to 50).  You can easily do this with your related items as well.

    In most cases, you wouldn’t show all of the related information of your jobs when just showing the list of jobs as you’re wasting bandwidth doing that.  You should only show data when the user specifically needs or asks for it.  For example, IMHO there’s no need for that 2nd foreach you have.  Instead, I’d list the jobs with the company name beside it and only show the list of products relating to a job after they’ve clicked the job for the details.  Otherwise, you have all this product information loaded in other jobs, when they might only care about the products of that one job.  You know what I mean?


    @OverZealous.com: Yeah, slicehost is nice and cheap for the amazing service they provide.  I’m starting off with a 256MB slice until I get everything setup and see what the performance logs say.  It’s good to be able to easily grow your slice as needed and their biggest one sure is meaty, lol!

    Although it’s very hands on and can take up a fair amount of time to configure and maintain, the level of control you get far out ways the downsides for me.  It’s still early days for me on there though so I’ll let you know how I find it in the longer run.

  • #325 / Dec 05, 2008 2:08am

    empoleon

    5 posts

    @stensi: I corrected my original post—you pointed out my method was missing a query I’d neglected to paste in:

    $job->product->get();

    You’re right on about presenting only the data the user has requested or needs at the time (i.e. removing the 2nd ‘foreach’ loop).  I was thinking a bit about those special people in the workplace who need ridiculously verbose reports and how Datamapper would manage such queries.  You reminded me that our server can manage the joins—I’ll crack on then.

    Thanks for your excellent explanation. It really cleared things up for me.

  • #326 / Dec 05, 2008 7:12am

    OverZealous

    1030 posts

    @stensi

    Bug in new save method:

    The new save method has a fairly serious bug.  Because it tries to save the object every time save is called - even when an object is passed in, new objects that are saved to the database get re-saved multiple times.

    Example:

    $user = new User();
    $user->name = 'Bob';
    $user->email = '[email protected]';
    $user->save();
    
    $group = new Group();
    $group->get_by_name('user');
    
    $user->save($group); // $user gets REsaved to the database here.

    Why does this happen, you ask?  It’s because the $stored array is populated with the original (all NULL) data.  However, the new object now has an ID, a name, and an email (in the example).  Therefore, it always appears to have changed.  Obviously, the $stored array needs to be updated after a successful save.

    The real issue, however, came when I tried to save something with NOT NULL datecreated and dateupdated fields.  For some reason, DM is trying to save them as NULL.  I’m still looking into this one.

    UPDATE: Found that sucker!  Your tests to see if the create_date and updated_date fields is still stuck on array_key_exists, from DataMapper 1.5.2!

    As for the other, here’s my fix, but it’s too frickin early, and I need some sleep, so I didn’t get a chance to test it thoroughly.  I think it’s still saving them multiple times.

    // immediately after insert or update:
    // populate $stored
    foreach($data as $field => $value)
    {
        $this->stored->{$field} = $value;
    }
    
    // only in insert:
    // Assign new ID
    $this->id = $this->stored->id = $this->db->insert_id();

    Also, I think you made a mistake in the test to see if the only changed field was $updated_date.  It’s inside the unset loop, and I think it’s supposed to be after that loop.

  • #327 / Dec 05, 2008 6:42pm

    stensi

    109 posts

    Ah, thanks very much for that!  You’re right on all accounts.

    Just to note though, when saving you can do both the INSERT/UPDATE of a new object at the same time you save a relationship onto it.  So, this line is not necessary:

    $user->save();

    Due to the fact you have this line:

    $user->save($group);

    With a properly working version of DM, there’s no issue with having both lines of save code there though, since the first would insert the user and the second would see the user itself has not changed and so only insert the group relationship (if any user fields had changed though, they would update at that time).

    You could leave both your save calls in there and add the following check before saving the relationship:

    if ($group->exists())
    {
        $user->save($group);
    }

    That’ll save on a bit of extra validation processing.

    I’ve fixed up all the issues you mentioned except for the NOT NULL/NULL thing which I’ll try to replicate now.

  • #328 / Dec 05, 2008 7:33pm

    OverZealous

    1030 posts

    The NULL bug should be fixed by changing the created/updated_date line.  It was just skipping them, so it saw them as NULL values.

    The reason I can’t do it all at once, in my code, is I do some pre-processing for security reasons.  I need to be able to save the data, verify it, save the next item, verify it, etc.  Actually, I’d almost rather have the option to disable the auto-save.

    Would it make sense to add a flag on the save method to skip saving this object?

    function save($object = NULL, $save_this = TRUE)
    {
        if($save_this)
        {
            // save this object
        }
        if( ! is_null($object))
        {
            // save relationship
        }
    }

    Also, I’m dynamically processing some JSON-encoded fields.  Once you leave the realm of normal HTML forms, things get weird. 😉

  • #329 / Dec 05, 2008 7:54pm

    stensi

    109 posts

    Ah, I was wondering why I wasn’t able to reproduce it, lol 😊

    I benchmarked the save to see the impact and, if there are no changes to the object between a save without a relationship and a save with a relationship, then there is no noticeable impact.  The comparison between the field and the stored field happens very fast and skips over the more involved validation processes, so yeah, it’s fine the way it is 😊

    I’ve fixed up everything now and have improved the use of the CI Form Validation library to instead load it normally rather than creating instances of it.


    _________________________________

    Version 1.5.4 has been released!

    View the Change Log to see what’s changed.

    In short, the stored values are now being refreshed correctly during save()‘s and a silly bug was fixed (thanks to Phil for bringing these to my attention).  There’s also a minor improvement to the use of the CI Form Validation library.

    Enjoy 😊

  • #330 / Dec 06, 2008 3:43am

    aztack

    2 posts

    Great Job stensi~~ That’s is what I need! Thanks!

.(JavaScript must be enabled to view this email address)

ExpressionEngine News!

#eecms, #events, #releases