Thread

@stensi:  Excellent work on the DataMapper library.  Since DM is PHP5 only, have a look at the latest KohanaPHP ORM class in the 2.3 trunk for some examples of ORM optimizations.  Some optimizations were recently implemented to reduce the number of queries when working with related tables.  DataMapper is a definitely a great extension to CI.

Datamapper seems great work indeed.
Too bad for me you’re using tight specs such as relationship tables. I understand that it makes things easier to develop, but I can’t change my database structure for this project. In my sense, an ORM should map the database structure, not the reverse, but anyway, keep the good work, I’ll sure use Datamapper on another project.

@phme

it think your right in essence that it makes it more flexible and therefore more practical when applying it to existing databases. Really though, the only addition DM might need to allow for non-5nf set-ups is a join_keys function. That way you could (if desired) run joins based on columns instead of join tables. I’m working on a function like this now, because some meta information i like to store with content shouldn’t need a dozen tables to support it.

imo, enforcing a specific database setup helps in that a lot of costly processes like running and analyzing tables can be forgone by assuming a strict db structure. And if you ever wish to think about scalability and performance, then any inefficiency in data retrieval will be scaled likewise.

@cc, thanks for the reply, I’d love to see this function if you’re happy with it.

I’m not sure I agree on your second paragraph: I’m not a ORM developer, but I’d guess that having an arbitrary rule for joins vs. one that is explicitly defined by the user (as in your join_keys function, I’d think) does not create much overhead. Same goes with the way foreign keys are named, and for that matter, the tables names themselves. But I might be totally wrong.

Conversely, a basic 1-n relationship represented through a relationship table is not normal (in the normalization sense): afaik, the fifth normal form is meant only for n-n relationships. This way does add some overhead—usually negligible, I agree.

The worst (not uncommon) case I see is the first basic relational structure example one is usually exposed to: the ‘customers’-‘products’-‘orders’ case, here translates in five tables and a sale action performs three writes on the database, since there is no mechanism to retrieve specifics of the ‘orders’ table from the system.

Granted, when we build web applications, we’re usually more concerned by reads than writes, and the join does not take much longer, but I still find it a tad shocking.

That said, I’m totally incapable of writing an ORM, even less so one that performs well, as I’m sure DM does, so I really should not even dare open my mouth 😊

I try to use the DM1.4.5 with the HMVC(ME).

The HMVC change the directory structure and got the error ‘file isn’t exist’
on line 194 in dataMapper.php

Is Maybe be fix on next version?

First of all, thank you stensi for the wonderful tool. I’ve been experimenting a bit with DataMapper 1.4.5 runing on CI 1.7 and have 2 issues.

First, let’s say we have the database following schema:

CREATE TABLE IF NOT EXISTS `users` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `name` varchar(40) collate utf8_unicode_ci NOT NULL,
  `type` enum('admin','user') collate utf8_unicode_ci NOT NULL
);

CREATE TABLE IF NOT EXISTS `posts` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `title` varchar(40) collate utf8_unicode_ci NOT NULL,
  `subject` longtext collate utf8_unicode_ci NOT NULL
);

CREATE TABLE IF NOT EXISTS `posts_users` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `post_id` int(10) unsigned NOT NULL auto_increment,
  `user_id` int(10) unsigned NOT NULL auto_increment,
);

INSERT INTO `users` (`id`, `name`, `type`) VALUES (1, 'John Doe', 'admin');
INSERT INTO `users` (`id`, `name`, `type`) VALUES (2, 'Billy Jean', 'user');

INSERT INTO `posts` (`id`, `title`, `subject`) VALUES (600, 'Post 1 title', 'Post 1 subject');

INSERT INTO `posts_users` (`id`, `post_id`, `user_id`) VALUES (20, 600, 1);

Now when we try this:

$user = new User();
$user->where('id', 1)->get();// get user by id 1
$user->post->get();// get related posts
echo $user->post->id;// PROBLEM - echoes 1 (user_id) instead of 600 (post_id)!

The problem is that $user->post->id returns the $user->id value instead of the $post->id value.

The second issue is getting id doesn’t work correctly on newly saved objects:

$a = new User();
$a->name = "Jim Beam";
$a->type = "user";
$a->save();// OK here - saves the object to the database with a new id
echo $a->id;// PROBLEM - echoes NULL!

I believe this is an issue with $db->insert_id not being saved correctly. Has anyone else encountered these issues?

Hi dedavai.  What type of Database are you using?  Also, what version of PHP are you running this under?

I was unsuccessful in trying to reproduce your issue.  In both situations I received the correct ID.

And just a quick update for all, the next version of DataMapper wont be long now.  There’s a lot of performance improvements as well some great new functionality on the way!

@stensi

Super excited to see the new update! i’ve been working out ways to improve performance as well. Thought i’d just toss out a few ideas i’m working out and see if you think any are good.

One thing i found is that running isset($array[$value]) is up to 10x faster than in_array($value, $array), so on construction i’m having all arrays converted to string index, and then using isset to check for values. Also i’m finding it handy in getting a specific field from the validation or relation arrays, as opposed to looping each time.

Also, i had replaced the changed_existing methods. I have the constructor creating a stdClass called ‘stored’, which just creates a copy of the values during a successful get or save.
That way to tell if a value as changed you just check
($this->{$field} == $this->stored->{field) you could use str_comp() too, i suppose. Also it doesn’t require additional queries.

I made a few changes to the _to_object() method too, esp with some additional join functions. one change seems to help a lot, especially when creating dynamic forms. i just set

$index = intval($item->id);
$items[$index] = $item;

so you can access a specific record in the all array with $this->all[$id], or even easily check if an id exists with isset($this->all[$id])

anyway, hope to see the new release soon, thanks for all the hard work 😛

CC

I’ll look at including those changes in the coming version since they’re all very good ideas, thanks! 😊

UPDATE: On closer look at the benefits of isset vs in_array, I’ve found that the performance difference is small enough as to be negligible. So, where it improves functionality, such as with giving the all list access by id and so on, I’ll switch to associative arrays and use isset but otherwise I’ll leave it as in_array since it provides for cleaner code.

cool, thanks! I’ll post some more as i figure this stuff out if you like. Here’s one more quick handy function i added, which copies over confirm (non-existing) fields automatically, when needed. Also in the config file i added $auto_copy_confirm = TRUE|FALSE, which calls it on a successful get()

public function copy_confirm_fields()
{
    foreach($this->validation as $field => $data)
    {
        $rules = isset($data['rules']) ? $data['rules'] : array();
            
        if( isset($rules['matches']) )
        {
            $match_field = $rules['matches'];
                
            if( ! empty($this->{$match_field}) )
            {
                $match_field = $rules['matches'];
                $this->{$field} = $this->{$match_field};
                $this->stored->{$field} = $this->{$match_field};
            }
        }
    }
}

I’m working on several other things with DM, too. Like i mentioned in a previous post, i’m sorta working out a framework to use with datamapper. So far I have a Datamapperlist, Datamapperform and Datamapperobject library set going. The object lib is an extender of DM, and those models are loaded into the list and form libraries, which automatically handle what they’re supposed to do. for example

$user = new User();

// Call where/join methods..

$list = new DataMapperList($user);

$list->get(); // sets order_by and limit/offset automatically, as set/stored in session vars

echo $list->get_pager();
echo $list->get_table(); // returns a sortable table of results, with formatted data & other options

// or alternatively..

$list->load_view($view_file); 
$list->get_list(); // returns sortable list with each result mapped to $view_file

As far as the form library, i think i may move all the validation functions into that, although that’s sort of getting away from what DM intended to be, but i suppose i’m trying to get DM to be ‘pure’ Database functionality and let the extensions and other libs to do the extra work only when needed. I dunno if that’s the best way, but it’s new territory for me so i’m giving it a shot. I also want to make a DataMapperUtility library, which can create join tables, edit relationships, or convert foreign key fields to join tables etc. I’ll post some more on them later when its all finished up.

edit: yeah i realize the boost in performance is small, and isset can be messier. I mainly just wanted to be able to access data directly without having to always run loops and stumbled upon the isset thing, so i figured why not use it if i converted the arrays anyway.  😛

cheers,

CC

Ah, brilliant. The copy of the corresponding field for the ‘matches’ rules is something I’d completely forgot about doing but now that you’ve reminded me, I’ve included a copy in both the get() and _related() methods.  I’ve simplified it into a few lines and am making it always do a copy since that would be the correct behaviour.  I can’t think of a situation where you wouldn’t want it defaulted to the matched value when loading an existing record.

Making the data easier to format when displaying in your HTML is something I’ve been thinking about since I read OverZealous’s comments on how he was doing it.

One of the neat new methods I’ve added is a magic get_by_{field} method. So for example, you can do:

$u = new User();
$u->get_by_username($username);

echo $u->username;

Which is the same as:

$u = new User();
$u->where('username', $username)->get();

echo $u->username;

Obviously, you can use the get_by_{field} method for any field the object has, including get_by_id($id).  In the same ease of use, I’m still figuring out the best way for an HTML display method. I was thinking something like this:

$object->display_{property}_as_{format}()

For example:

$p = new Product();
$p->get_by_id($id);

echo $p->name;
echo $p->display_cost_as_currency();
echo $p->display_description_as_html();

How’s that sound? If you can think of a better way for displaying the property values, let me know!

Alternatively, but a lot more work, would be to create another class, say “Property”, and have each field value stored as a Property object.  When doing:

echo $p->cost;
echo $p->description;

It would simply echo the value as normal. But I’d add extra functionality in the Property class to allow:

echo $p->cost->currency();
echo $p->description->html();

Basically, each property would have several static display functions.  Again, this would be a lot more work but it might make things more flexible in future.  One downside is there’ll be a slight increase in memory usage due to the additional Property objects.

Thoughts?  Which way would you prefer?  Also, can you see any other benefits or uses from having a Property class?

$object->display_{property}_as_{format}();
or
$object->{property}->{format}();

UPDATE: Just to clarify, I don’t mind if the preferred choice is the one that requires more work.  When adding new functionality, my main considerations are ease of use and extensibility.  It doesn’t matter to me how much work is involved, although it will mean you might have to wait longer 😉

those two solutions sound great. I’m guessing you’re talking about using the magic __call() method, no? Some people think its slow, but i think the ease of use for programming could be worth a small performance hit, which turns out isn’t super bad (Here’s some results on it)

i started creating a DataMapperField class, which would act as a factory to create objects based on field - type - value arguments. Each object type would extend an interface or abstract class with similar static methods

$object->set_value()
$object->get_value()
$object->display_value()
$object->form_element()
etc..

Then at some point you could turn each field into an object of the same name, and do exactly what you have in your second example. But yeah it is a lot of work! I haven’t even gotten deep into writing it and i really like the ease of the first example.

so if i were forced to personally select an approach i’d give the magic method a first shot. Anyway, it’s easier to write i’m sure and nothing would stop us from trying out the object creation method at a later date and benchmarking them side by side.

it would be great to have these magic methods

get_by_{$field}
get_{$field}_as_{$format}
get_{$field}_input (form element)

the one thing that comes up though, is how to handle related objects which are accessed just like fields, $object->{$value}. So if you tried, say get_{$related}_as_{$format}, how would that work?..

I’ve added a $primary_value varible to DM, defaulted to id and specified in a model/extend (picked this up from the kohana orm class, thx neovive!), and also added this tiny function

function get_value()
{
    $value = $this->primary_value;    
    return empty($this->{$value}) ? FALSE : $this->{$value};
}

so that helps a bit with dynamic html generation, which should handle relations like fields sometimes. Also, you could have a dynamic form dropdown function with (i’m writing this ad-hoc)

function get_dropdown( $limit, $offset, $where = NULL )
{
     $this->load->helper('form');

     $this->db->select('id, '.$this->primary_value);

     if( ! empty($where) )
     {
          $this->where($where);
     }
     
     $id = $this->id;

     $query = $this->db->get($this->table, $limit, $offset);

     return form_dropdown($this->model, $query->result_array(), $id);
}

that provides a base for extending models, which could always have the option of overloading the functions.

As far as more related model functionality goes, i’m working on an overall table-alias/field-alias scheme to allow for efficient SQL using multiple joins without risking table or field ambiguity. Also as i’m working with a join_key function, it can turn foreign key fields into referenceable data ie. SELECT user.id, user.friendID, friend.username as friend_username FROM ‘{user_table_name}’ as user, left join (user_table_name) as friend on friend.id = user.friendID

anyway the relation issue is another topic.. would love to chat with you more about it too.

cheers,

CC

On the “$object->get_{$related}_as_{$format}()” thing, that shouldn’t be a problem since if you’re displaying related data you shouldn’t do it that way, but rather as “$object->related_object->get_{$field}_as_{$format}();”. For example:

$u = new User();
$u->get();

// Display username as HTML
$u->display_username_as_html();

// Display related group name as html
$u->group->display_name_as_html();

For now, I’m still undecided.  I think in the end for me, it will depend on how much functional benefit there would be in turning properties into custom property objects.  Right now I’m brainstorming what possible methods I would use on them, besides display methods, but without going overboard.

The primary value thing would be handy for general HTML type of things, such as with your drop-down method.

The related join stuff is something I’ve set aside to look at after I’ve completed all other tasks, since it usually ends up being a bit of a headache figuring out, lol. I’ll definitely be picking your brain about it when I get up to that point! 😊

I have one big concern about integrating these functions directly into DM - unnecessarily loading a lot of class data if it isn’t needed.  If I’m just looking up some value - and not displaying it - I don’t want to load in the display or editing classes.

In my case, I’m mostly sending my data over JSON-formatted data, so there actually is no display or formatting code being used!

The reason I chose my method - using the “->v->${field}” or “->e-${field}” methods is that it allowed me to load up the class when it was needed, but completely silently.  Also, this class only needed to be instantiated once each per object.  not perfect, but not too bad.  I might actually try to move more of my code into shared objects, eventually.

Of course, you could make this work just fine with stensi’s method - using magic get_${field}_as_${type} - you would just have to load a rendering or editing class in the background.  I think I have a way of combining the methodologies.

First, I would offer the magic fields display_${field}_as_${type} and edit_${field}_as_${type}, as stensi mentioned

Second, allow the special forms view_${field} and view_${field}_as_default (same for edit_) to exist, and to choose the type based on the (optional) $type parameter of the validations field, or default to simply calling htmlspecialchars() on the field.  For editing, it should simply place the htmlspecialchars()‘ed in a plain <input type=“text” />.  Also, the virtual properties (not methods) $view_${field} and $edit_${field} should call view_${field}_as_default() behind the scenes.  This is very clean to read and write:

// example formatted this way because the website won't properly render <?= ?> tags
echo $user->view_firstname;
echo ' ';
echo $user->view_lastname;

Third, load a single class in that can handle the rendering, but only upon first access.  The best case would be to store this class as a static object within DataMapper, or within each class, to reduce overhead.  The methods would, obviously, be named after the ${type}.

Fourth, this class needs to be able to be overridden.  In fact, I think it would be best if the default class was extremely basic - maybe only offer the defaults listed above.  The name of the class could be described the way updated and created fields are, within the DM config, or the way model or table are, within the object.  You could label it $field_displayer and $field_editor.  Then, when instantiated, you could do:

if( ! isset( ($model)::$_dm_field_displayer ) ) {
    if( isset($this->field_displayer) ) {
        $this->load->library(strtolower($this->field_displayer));
        ($model)::$_dm_field_displayer = new $this->field_displayer;
    } else {
        if( ! isset(DataMapper::$_dm_field_displayer) ) {
            $model = // get model from datamapper config
            $this->load->library(strtolower($model));
            DataMapper::$_dm_field_displayer = new $model
        }
        ($model)::$_dm_field_displayer = DataMapper:$:_dm_field_displayer;
    }
}
// now look up function in ($model)::$_dm_field_displayer

This also allows for methods that need or can take arguments, such as display_${field}_as_limitedlength($length, $use_ellipsis)

Finally, I agree here with Stensi that the related objects should not be embedded within DM.  In the end, the question is, how much should you combine your View with your Model and Controller.  Of course, you really should keep them separate.  That’s why I prefer, somewhat, my method of using a virtual property.

As for me, I probably won’t use it at all unless it is completely overwriteable.  All of my editors rely on the Dojo JavaScript toolkit, and both my viewers and editors depend on localization code and the structure of the DM object.  For example, I print out the complete XHTML form row, including a properly linked label tag, error messages, etc, from a simple ->e->name.  It helps to keep my code as light as possible, and keeps 90% of my formatting in one spot.

Hopefully that is some help.

PS: I actually prefer the term “viewer” instead of “displayer”, and therefore the function name view_${field}_as_${type}...  But that’s not terribly important.

Now that I’ve seen it, I would prefer the naming of view_… over display_… since it fits in better with the names already used by CodeIgniter.

It would be easy to provide “edit” versions using the form helper to spit it out as a form input.  I’m assuming what you mean is that the view_… would be different formatting types (currency, phone_number, html) and edit_… would be different form input types (checkbox, text, dropdown)?  Is that what you mean?

So, for example:

View:

// $object->view_{field}_as_{format}();

$u->mobile = 1234567890;
$u->active = TRUE;

echo $u->view_mobile_as_phone_number();
echo $u->view_active_as_image('gif');

// This might spit out:
1234 567 890
active_true.gif

Edit:

// $object->edit_{field}_as_{type}();
echo $u->edit_phone_number_as_text();
echo $u->edit_active_as_checkbox();

// Phone would have a text input
// Active would have a ticked checkbox since active is TRUE

The above is theoretical. Just trying to clarify preference.

If I was to go ahead with this stuff I would make a method for each type of “format” and “type” option, so they can be overridden.  As all formats and types would be standard, it would be stored as a single static variable for all DataMapper objects.