Thread

Better handling of 64KB+ content fields

July 06, 2017 2:30pm

Subscribe [2]
  • #1 / Jul 06, 2017 2:30pm

    Paul Bailey

    44 posts

    Yesterday, after 10+ years of using EE, I fell foul of the maximum content field size, which I think is 64KB, and undocumented. This was in 2.11.3, in which fields larger than 64KB seem to be silently truncated, potentially losing data. In my case, the field had been edited enough times that data was actually lost on the server, and had to be recovered from my test set-up.

    Losing data silently like this isn’t good. I’d have expected at least a warning. But since EE2 is on its way out, I’m not going to whine too much about that.

    I just deliberately pushed EE3 (3.5.8) in the same way, though, and it doesn’t fail much less clumsily, giving a low-level SQL error and dump of the field:

    SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column ‘field_id_5’ at row 1:

    That sort of works as a warning that something has gone wrong, but it’s not a tidy way to catch or handle the problem.

    In addition to gentler error-handling in this situation, it might be nice to have a display in the editing screen which gives a sense of how close a field is to its practical limit. Something like: “2KB used of 64KB”.

    Thanks for considering.

  • #2 / Jul 06, 2017 2:35pm

    Derek Jones's avatar

    Derek Jones

    7176 posts

    I agree with you that we should trap content that would exceed the schema’s maximum size instead of letting MySQL barf an error. But can I ask what type of content you are storing that you are placing 64k+ into a single field? That seems very counter to the purpose of custom fields and using a CMS.

  • #3 / Jul 06, 2017 5:19pm

    Paul Bailey

    44 posts

    Derek:

    But can I ask what type of content you are storing that you are placing 64k+ into a single field? That seems very counter to the purpose of custom fields and using a CMS.

    It’s a directory of people that’s grown very quickly over the past couple of years. It definitely should be reworked somehow, but it’s one of those situations where there are also advantages to having the content in a single place, so there hasn’t been any strong reason until now.

    I wasn’t necessarily thinking of just my situation, though. In the general case, it’s not hard to imagine writing that easily fills 64KB — long-form journalism or short fiction, for example. These aren’t pieces that would break naturally into smaller chunks. With even simple markup, 64KB is about 8000-10000 words, which isn’t unusual in a Medium-like environment. I have no idea how rigid the 64KB limit is, but whatever the limit is, it should be made more apparent to the user.

    (Aside! The box I’m typing this reply in tells me how many characters I’m allowed, and how many are left. That’s exactly the sort of thing that I’m thinking would help the EE back-end.)

ExpressionEngine News

#eecms, #events, #releases