Thread

I actually prefer the same “types” to be used in editing and viewing.

For example, if I am working with a date field:

// in the Post class
$validation = array(
    'editdate' => array(
        'label' => 'Edit Date',
        'rules' => array(...),
        'type' => 'date'
    ),
    // ...
);

// ------------------

// in a view
$post->view_editdate(); // outputs the field as a formatted date
//or
$post->edit_editdate(); // outputs the field in a textbox, formatted for editing, maybe including a popup calendar for selecting dates.

For many formatting types, the editor will just defer to the default single line editor.

For checkboxes, I actually used the phrase “checkbox” both times, since I wanted the field to be rendered as a graphic of a checkbox selected or not:

// in viewer
// renders a field as a checkbox
function checkbox($object, $field) {
    $value = $object->${field};
    if($value) {
        echo 'images/checkbox_checked.png';
    } else {
        echo 'images/checkbox_unchecked.png';
    }
}

For selection lists, I either passed in a list of values or I looked up a standard array within the object

// in viewer
function selection($object, $field, $values = NULL) {
    $value = $object;
    if(is_null($values)) {
        // look for values array in both object and class
        if(isset($object->{$field.'_values'})) {
            $values = $object->{$field.'_values'};
        } else if(isset($object::{$field.'_values'})) {
            $values = $object::{$field.'_values'};
        } else {
            show_error("No values defined");
        }
    }
    return htmlspecialchars($values[$value]);
}
// edit method is similar, or shares code with a common function to get the array

This allowed me to look up how to render the field for editing or viewing using $this->validation[$field][‘type’].

I also often like to have several options.  For example, the select list above could be reformatted as a list of radio buttons, if it made sense.  My code has a function called list() that automatically switches to radios for less than 4 items.

Just some ideas, referencing some of the problems I have had.  I’ve actually spent a ton of time refining my Html_viewer and Html_editor classes (that I believe you have a copy of, stensi).  Even though they are formatted for Dojo, they offer a lot of code for formatting a variety of fields.  For example, Dates, Times, and Timestamps can be formatted as ‘SHORT’, ‘MEDIUM’, or ‘LONG’, or custom formatted using the strftime() function.

Probably the biggest concern here is that every user of the system is going to want different formatting for their output, both on view and edit.  This is why I am not sure how much you want to include.  I think that providing a skeleton for the viewer and editor would be more useful, as it would be a guide for new users on how to add functionality.  You can see just how different a simple thing such as a boolean/checkbox field could be rendered (various images, true/false, yes/no, on/off, or even more specific, like lost/found)!

Adding the type setting in the validation array is something I’d probably do.  I’m using the field_data() method to get the fields of a table now, instead of list_fields(), and that comes back with additional meta data, such as the type of each field. So, if a type does not exist in the validation array, I can fall back on the type listed in the meta data.

I’d try not to go as far as giving set options on how things get formatted, such as SHORT/MEDIUM/LONG for dates.  The developer would need to supply the formatting information but I would at least have a default, and probably add an additional setting in the validation array for a developer specified default, such as format.  For example, formatting by date, the developer would specify the format in the same way you do with PHP’s date function:

// If we had a "type" set in the validation array for the "created" field
// and it was of type "date", this would output the default format for the date
// and if we had a "format" set as well, it would use that specified format
echo $u->view_created();

// Defining "type" and "format" in the method call
// this would outputs "created" as a date in the specified format
echo $u->view_created_as_date('F j, Y, g:i a');

This is all something I’ve only started looking at today so I’m not sure if I’ll be doing anything on it in the coming version.  I’ll mess around with it a bit tomorrow and see how far I get.

UPDATE:

Just looking at the other idea, of having each property turned into an object via a very small property class, I was thinking of just doing something simple like making it use any loaded helper or PHP function that accepts one or more parameters and apply the result to itself. For example:

$u = new User();
echo $u->username->htmlentities();

// Same as doing:
echo $u->username = htmlentities($u->username);

You could also chain it:

echo $u->username->trim()->htmlentities();

// Same as doing:
echo $u->username = htmlentities(trim($u->username));

You’d only really want to use prepping type functions.

For those that accept more than one parameter, it would only work correctly “out of the box” if the first parameter is the value being used (ie, the property value). So unfortunately, since the date function has the first parameter as the format and the second as the value being used, it can’t get used as is.  I’d need to make a mini-override that passes through the parameters swapped around, so I could still make it work.  An example of one that would work “out of the box” with multiple parameters is str_pad:

echo $u->username->str_pad(10, "_", STR_PAD_BOTH);

// Same as doing:
echo $u->username = str_pad($u->username, 10, "_", STR_PAD_BOTH);

Again, these are just ideas I’m throwing around for discussion.  I’ve made no decision yet on what I’ll be going ahead with as far as all this “formatting values” goes.

I agree wholeheartedly that any additional classes should be loaded only as needed. The great thing about using the magic methods though is that this can be achieved regardless of how a viewing or editing class is implemented. Whatever can be done to reduce the size of the classes, yet keep with good clean code is the way to go. It seems like we are all taking slightly different approaches but trying to achieve the same thing.

right now i’m actually trying to strip datamapper of everything besides database functionality (yes, including the validation), and then the other classes i’m creating are loaded explicitly in the controller, and act as ‘clothes’ for datamapper.. so when you call up the listing class and load the datamap into it, $datamap->{value} becomes $list->{value-formatted}, or $form->{value-input}, but again it’s only loaded when called. That’s similar to the $model->v->{$value} implementation you have, just not using the __get() magic method. I know that having the validation within DM is great, i just think that in most cases the only time you really NEED to validate is when dealing with user input, ie forms, and so it’s handled with the form class for DM, otherwise many times the values are set by code and don’t need to be validated to be saved (making for faster processing).

Also, i’m still working out the form generation for a datamap, and in some instances i’m adding another element to the validation array, ‘conf’, which holds whatever necessary config data i need for processing. This is where i keep the lists of say a select field

'status' => array(
    'field' => 'status',
    'type'    => 'list',
    'label' => 'Status',
    'conf' => array(
        'list_values' => array('1' => 'Active','2' => 'Inactive','3' => 'Suspended','4'=>'Pending'),
        'list_default' => '1')
    'rules' => array()
),

So when i do create the form element for that field, it knows to use a dropdown by its type (list) and then just grabs the list from ‘conf’ and inserts it directly into a form_dropdown(), also you can imagine that you could add css classes to each value in conf. That’s all handled by the form class.

As far as embedding relations in DM, if we do plan on implementing effecient sql using joins then DM should at least be able to do that and set them up. I actually overwrote the get and _to_object methods to try to clean up my own code. And in the form class i’m using, relations are validated as well, and also some have their own config (how else would you specify that a certain model MUST have one kind of relation?)

var $has_one = array(
    'userclass' => array(
        'label' => 'User Class',
        'rules' => array('required')
    )
);

Also, i’ve built two special models, ‘document’ and ‘photo’, which can’t be validated within the post array and need special handling/configuration. So in a model that relates to a pdf:

var $has_one = array(
    'document' => array(
        'label'     => 'Manual PDF',
        'conf' => array(
            'upload_path' => '../content/manuals/',
            'allowed_types' => 'pdf',
            'max_size' => 8000,
            'encrypt_name' => FALSE,
            'remove_spaces' => TRUE
        ),
        'rules' => array('required')
    )
);

Then the config is passed to the document model, and $document->upload() is called and either returns TRUE or passes any errors back to the form class for display. Having the photo model is turning out handy indeed as i can specify thumbnail size, specific proportions and size per each model that has one, plus i am working on functions for editing photos, like cropping, resizing or filtering. So while datamapper shouldn’t necessarily be aware of or handle its relations on it’s own, i think they can benefit a lot from more additions in the models themselves.

anyway, i’m glad we’re all collaborating with ideas for DM. I am myself by no means an expert on any of this stuff.. I can tell you i’ve been working hard with it for the last 2 months or so and it can be very frustrating! But i am excited to see what stensi comes up with for it, and I hope all our ideas are helping him.

thx,

CC

I like the idea of storing the array of values within the validation array - but it doesn’t allow for statically stored values (except, that’s being fixed in the next generation of DataMapper).  If the values are not stored statically, every instance of the class will have a copy, leading to unnecessary RAM usage on large data sets.

I also like the idea of adding rules to the $has_one and $has_many arrays.  In fact, I’m wondering if the validation array should just be combined into the $fields array, then all three would be more consistent - all would be associative arrays combining the name of the field with field properties.  That’s a pretty big change, I realize!

class User extends DataMapper {
    $has_one = array(
        'address' => array(
            'label' => '@user_address_label', // possible method of looking up labels using localization
            'rules' => array('required')
        )
    );
    $has_many = array(
        'phone' => array(
            'label' => '@user_phone_label',
            'rules' => array('required')
        )
    );
    $fields = array(
        'firstname' => array(
            'label' => '@user_firstname_label',
            'rules' => array('required')
        )
    );
}

(Don’t forget that you can easily check keys using array_key_exists() - it’s basically the same as using in_array(), and just as easy to read.)

I’m not sure there is a good reason to push the values array (and other config items) into yet another array (‘conf’).  Instead, I would just place configuration details right within the main $fields / $validation array.  It’s one less layer of abstraction, and one less hash lookup.  That feels like a personal preference, though.  I can see how it is easier to read with the conf array.

The way we’re going, a DM class will be completely declarative!  You can define most fields in the arrays at the top, with no methods or class fields needed.

I still like the validation stuff, but that’s because my application has 95% user-input, so it wouldn’t provide much benefit for me.  I wonder if, instead, it would make sense to have a function that saves the data without validating.  Same basic benefit, but without the overhead and complexity of separating validation from DM.  Maybe just add a parameter:

save($object = NULL, $validate = TRUE) {
    // ...
    if( ! $validate || $this->validate() ) {
        // ...
    }
}
// ...
$obj->save(NULL, TRUE);

All good stuff!  Poor stensi 😉

Hi all,

I’ve started a new project, and I am using Datamapper with HMVC together. It’s compatible?
I have an strange issue with DM 1.4.5…

User model
————————————

class User extends Datamapper
{
    var $has_one = array('province');
...
}

Province model
————————————

class Province extends Datamapper
{
...
}

Inside a controller I can get user<->province relation with no problem:

$user = new User();
$user->where('id',5)->get();
echo $user->name; //OK
echo $user->province->get()->name; //OK!

But, inside a module controller (module dashboard, controller profile) I’ve been trying the same, and echoes NULL:

$user = new User();
$user->where('id',5)->get();
echo $user->name; //OK
echo $user->province->get()->name; // NULL ?

Is anything related about autoloading (function autoload inside Datamapper class), because of only is searching inside APPPATH.’/models’ ?

Thank you for this great library!!

hehe, i know.. i hope we aren’t pulling him in too many directions. I’m gonna keep working on this stuff before posting too many more ideas. I think i do also need to get up to speed on static variables and methods, to allow for better memory usage. work work work!

I know that adding more config data to the models is going to increase memory usage, and i don’t necessarily like it per se, but so far it seems to be working for me. It would be nice if DM could even parse the table columns themselves. So for instance, if you have a field defined as ENUM, which naturally becomes a dropdown, perhaps DM could read that and create a static array containing it’s values. Although the only thing i found so far Here uses regular expressions, which are rather slow..

I did start using array_key_exists($value, $array), but even that seems to be slightly slower than using isset(), as isset is a language construct and doesn’t need to lookup the function hash, and doesn’t trigger errors, only returns true/false. Here’s an article comparing the two (btw, i try to take everything i read online with a grain of salt) I mean, the performance boost is really pretty negligable and it really could come down to personal preference. The one nice thing i DO really like about it though, is you can check a variable to any depth in an array, so you can do, for instance

isset($this->validation[$field]['rules']['matches']);

instead of having to do

if( array_key_exists($field, $this->validation) )
{
    if(array_key_exists('rules', $this->validation[$field])
    {
       if(array_key_exists('matches', $this->validation[$field]['rules'])
       { 
          return TRUE;
       }
    }
}

because if any of the keys to get to ‘matches’ didn’t exist, or weren’t arrays it would trigger an error.

I do like your idea of having the option to bypass validation. I was very reluctant to remove it from DM, and i’m not suggesting stensi do that either. Yeah much of the datamap input is done through forms which are validated, but some stuff is automatic, and other fields are strict (like a status list), which are often individually toggled, and their values don’t need it really. But again, it depends a lot on how its being used, and which way makes one more comfortable.

cheers,

CC

Lots of great ideas there, most requiring a lot of work, lol! 😉

I’m thinking I might have to make a release with all the current changes I’ve done, which is a lot of optimisations and performance benefits as well as some new methods, before going ahead with any of these possibly drastic changes.

Being able to validate relations is a bit of a tricky subject. At the moment, you have to save a record first, before you’re able to save a relation. For example:

// Get moderator group
$g = new Group();
$g->where('name', 'Moderator')->get();

// Save a new user
$u = new User();
$u->username = 'foo';
$u->password = 'bar';

// Save new record
$u->save();

// Save relationship
$u->save($g);

As you can see, I have to save the user first before I can save the group relationship.  If I have a validation rule that states a user can’t be saved without a valid group relationship, I’d never be able to save a new user. So, that would require a reworking to allow the possibility of saving a relationship at the same time as saving a new object.  For example:

// Get moderator group
$g = new Group();
$g->where('name', 'Moderator')->get();

// Save a new user
$u = new User();
$u->username = 'foo';
$u->password = 'bar';

// Save new record as well as relationship
$u->save($g);

It’s do-able.  Obviously when deleting a relationship, I wouldn’t want it to delete the objects record as well. On save, I’d have checks like:

- if record does not exist yet (empty ID), save both it and save the relationship
- else if record exists, save any changed fields and save the relationship

If you were saving a relationship on an existing item, would you want it to update any changed details on the record at the same time? And if the record didn’t exist you’d want it created? If so, then there’s no issue and I can go ahead with putting this in.

Alternatively, but not as clean, I could have a second parameter, so the developer has to specify the saving of both (not my preference):

$u->save($g, TRUE);

I’ll try to find time to work on the above today, and how to validate relationships, since this is a pretty important requirement.

Merging the validation array into the fields array would make sense for the most part, but yes, a pretty big change.  I’m dreading more having to update the documentation than modify the code for that though, lol, since the code would be easy to do.  Also depends on how big of a headache this would cause people currently using it, in having to change all their models.  Any drastic changes like this I’ll have to think about.

At some point I will be adding a type property to the validation array, as well as one possibly called options which includes the list of possible values a user could select, in say a drop-down list. That would mean I’d obviously have to have a default setting too, so the default option could be selected.

Lots to do and think about!

@Muser, @sankai: Sorry, DM is currently not setup by default to work with HMVC.  You’ll need to modify the autoload method in DM for it to be able to search the HMVC modules path.  I expect that you’d just need to modify the $path setting.  Search for the following:

// Prepare path
$path = APPPATH . 'models';

And I’m guessing for HMVC you’d change it to:

// Prepare path
$path = APPPATH . 'modules';

Thank you stensi!

Just another question:

1 user has only 1 role
1 role is used for many users

Having two tables…
users(id,name,role_id)
roles(id,name)

Is this possible to get it work with DM now?

Or I must to have 3 tables?
users(id,name)
roles(id,name)
users_roles(id,user_id,role_id)

At the moment you’ll need the 3 tables but with the joining table named in alphabetical order. Example:

users
id
name

roles
id
name

roles_users
id
role_id
user_id

To setup the relationship in the model’s as you’ve described (one to many):

In your User model:

$has_one = array('role');

In your Role model:

$has_many = array('user');

UPDATE:

Before, when I was talking about looking into saving relationships at the same time as saving a new record, well, this is now possible in the coming version.  You will also be able to update an existing record (it will update changed fields only) at the same time you create or update a relationship on it.

What I’m trying to figure out now is the validation, since that still goes in the same order as I mentioned… I’ll most likely have a separate validation process for relationship rules that get run when saving a relationship.  The validation array would simply have the rules for related items added as normal.  For example:

A User object must relate to a Group object.  The last entry in the validation array is the rule that a relationship to a group is required.  DM will be able to figure out that “group” is not a related field, not a field of user so it can validate it as a relationship.

User

var $validation = array(
    array(
        'field' => 'username',
        'label' => 'Username',
        'rules' => array('required', 'trim', 'unique', 'min_length' => 3, 'max_length' => 20)
    ),
    array(
        'field' => 'password',
        'label' => 'Password',
        'rules' => array('required', 'trim', 'min_length' => 3, 'max_length' => 40, 'encrypt')
    ),
    array(
        'field' => 'confirm_password',
        'label' => 'Confirm Password',
        'rules' => array('required', 'encrypt', 'matches' => 'password')
    ),
    array(
        'field' => 'email',
        'label' => 'Email Address',
        'rules' => array('required', 'trim', 'unique', 'valid_email')
    ),
    array(
        'field' => 'group',
        'label' => 'Group',
        'rules' => array('required')
    )
);

Besides the required rule, I’ll be adding a couple of other rules so you can specify an upper limit on how many relations each object can have (obviously this would only be intended for One to Many or Many to Many, since One to One is already restricted).

For example, you might have an Author object that can have one or more Books, but you want to restrict it to be from 1 to 20 books.  You’d do:

Author

var $validation = array(
    array(
        'field' => 'book',
        'label' => 'Book',
        'rules' => array('required', 'max_size' => 20)
    )
);

Hmm, there’s also the matter of not allowing you to delete your last relationship, such as with the example that a user must relate to a group, so that you instead must update it or delete the user to delete the relationship.

yeah i’ve run into that bump in terms of validating relationships on ‘new’ models. basically i open the form class with $edit_mode = ! empty($this->model->id), and then that can be used as a switch for whether to test relations. But still, in order for a required relationship to be properly saved, they both need to exist and both need to be saved simultaneously.

At the moment i have it set up so you can only select from pre-existing relations, and for any non-file model that’s a form element (typically dropdown) with the name {$relation} and with <option value=”{$id}”>{$primary_value}</option> as the standard. So, then i process forms by collecting all possible values from the fields/relations in post array, also checking for file relations, and inserting their values in arrays then running validation on them. If it all checks out then copy all values and proceed with a save (where needed).

It gets a bit trickier with ‘many’ relationships, in that i have them producing form elements with the name {$relation}_{$id}, and it scans for changes in those, but you don’t want multiple select lists and allow the same relation chosen twice, plus the old relationship has to be deleted on an update. It get’s even hairier when dealing with files, heh..

I think i may build some more common functions like get_value() for generic datamapper usage, such as collect(), for grabbing data from post/get/files, which i have running in my form class.

also, one other thing i’m trying, and i’m kinda just curious whether you think its a good idea or not, is I replaced the $related array which just has the model, table and id, with an actual object reference to the model that set it. Then you could do a save_to_parent() method or something.. i dunno it seems helpful to have a two way road between models and their relations, plus this may help with form collection, validation and saving, too. But can you think of any reason why that would affect performance if done correctly?

@Muser Dang it, i haven’t even gotten into modules yet, but HMVC seems pretty cool.. let me know if that solution works for you.

thx,

CC

Good news. I’ve finished implementing the validation of relationships and being able to save a new or existing object optionally with relationships at the same time. Due to the complicated nature of relationships, there’s a different type of validation rule you can setup.  Normally you’d have an underscore followed by the name of the validation rule: _{rule}(), for example “_encrypt”, but for related validation rules it is an underscore follwed by related and then the name of the validation rule: _related_{rule}()

Currently there are only 3 inbuilt related validation rules, as I couldn’t think of any others that would actually be necessary:

_related_required (used as ‘required’)
_related_min_size (used as ‘min_size’ => 123)
_related_max_size (used as ‘max_size’ => 123)

The only thing is, there’s no validation when deleting an object, so unfortunately, I haven’t implemented a check when delete is called to see if deleting the object will drop it below the min_size or if there would be no relationships left, thus required fails.  I might add that in since it makes sense to.

Everyone will have to make absolutely sure they’re checking the boolean return value when saving and deleting so you’re handling it correctly if something isn’t saved or deleted due to it failing validate(), which you should be doing anyway 😛

@commandercool: Replacing the $related array is an awesome little improvement.  I’ve gone ahead and done this, replacing it with a $parent property that is a reference to the parent object.  Makes the code look much cleaner and no, there shouldn’t be any performance issues from doing it this way.

I have a little bug for version 1.4.5

I use Codeigniter 1.7.0 and MySQL 5.0.51b when i use
$u = new User();
$u->where(‘username’,$this->session->userdata(‘username’))->get();
$u->bookmark->get();

that bookmark and user has associated * to 1 but i get this

A Database Error Occurred

Error Number: 1054

Unknown column ‘bookmarks.*’ in ‘field list’

SELECT `bookmarks`.`*` FROM (`bookmarks`) LEFT JOIN `bookmarks_users` ON `bookmarks`.`id` = `bookmark_id` LEFT JOIN `users` ON `users`.`id` = `user_id` WHERE `users`.`id` = 3

i just notice that if i remove some symbol it can run on mysql like

SELECT bookmarks.* FROM (`bookmarks`) LEFT JOIN `bookmarks_users` ON `bookmarks`.`id` = `bookmark_id` LEFT JOIN `users` ON `users`.`id` = `user_id` WHERE `users`.`id` = 3

I don’t know why i have this bug can you give a solution for this version. I don’t like the old way of 1.3.4 that works fine for me

Edit. Well 1.3.4 is not work fine :’( so how can i fixed this

i have this table
bookmarks
users
bookmarks_users (id,bookmark_id,user_id)

Edit2. I just test again with Codeigniter 1.6.3 with DataMapper 1.4.5 it works fine.

@DominixZ: The issue you’re having is due to a bug in CodeIgniter 1.7.0, not DataMapper.  The CodeIgniter Team have fixed it in the SVN though.  You’ll need to download the latest version of the mysql_driver.php file.

I’m including instructions for this in the coming version of DM, until there’s a new CI release that includes the fixed driver.

_____________________________________________

Version 1.5.0 has been released!

View the Change Log to see what’s changed.

There’s a lot of performance improvements as well as several new methods and greater functionality. I recommend taking a look at the Change Log for a summary but definitely re-read the DataMapper User Guide.

Feedback and bug testing is welcomed!

Enjoy 😊

Note that this version does not include any of the HTML display functionality that was previously discussed, as that will be coming in a later version.

Also, I still want to include the ability to do better “related” queries, such as related_where(), related_like() etc. but I haven’t had time to properly look at this yet.

Performance decreased by 8-9% ;[

Test query below;

foreach($c->project->where('name', 'Sqripted')->get()->milestone->where('version', '1.0alpha')->get()->task->where('name', 'Performance Bug in Assets')->get()->entry->get()->all as $f)
{
    echo $f->comment;
}

Under 1.4.5, execution was an average of 0.0297s, 1.5.0 is up to 0.0323s.

That said, this is a great library and will be incredibly handy for a big project we have coming up.

Interesting.  What sort of volume of records is that with?  It shouldn’t be slower as there are a lot less foreach loops taking place as well as many other speed improvements, and generally much better code.

I’ll setup a comprehensive test suite to stress test both versions and let you know the results.  I wonder if it’s anything to do with the use of static variables or perhaps the magic __call() method.