Thread

Hello,

I employ the black and white list module to try and help with the fight against spam

and have noticed that when adding additional ip addresses to the blacklist folder that having done so and receiving the message

Blacklist updated successfully

when scrolling down to the bottom of the blacklist list the last ip address on the list is missing the full IP address as follows:

83.147.141.233
83.149.74.21
83.150.83.60
83.167.106.78
83.

thank you for any observations…

Kelli,

I’m sorry to see this is happening, let’s see what we can do to get it sorted out. When you say “folder” are you referring to the list in the EE Control Panel? Can you share with me what version of ExpressionEngine is showing this behavior?

Cheers,

I don’t know if it’s still a problem with 2.x but I know there used to be problems with a large amount of IPs stored in that module due to them being stored in the exp_sites table. Might be that there are too many IPs listed in the module perhaps?

Hi,

Currently using EE version 1.71 and blackwhite list module 2.0

And the htaccess file That the black/ White list module writes to, amounts to 197kb’s

I didn’t realise there was a limit to the number of ip addresses stored within the blacklist.

I have calculated there are currently 4565 Ip addresses within the Blacklist, 799 URLS, 239 useragents

Is there a way to increase this?

Hi Mark, Just had a look in the mysql db and found them listed in the exp_blacklisted table

Thank you

Hiya,

Yep I do believe that there may well be a limit on how many IP addresses are allowed in there and once it reaches the limit of that field it will start to cut off.

The field is stored as a TEXT field in the database which means that it has I believe a limit of 65,000 bytes which may well be why you are experiencing this problem.

The only way around this would be to change the type of field in the database so that it can hold more information although I’m not sure if that’s safe to do or not or whether it would then still work within ExpressionEngine. I don’t think it would cause a problem although one of the support crew will be able to answer that better.

Best wishes,

Mark

Hi Mark,

yep reviewing the tables using Sequel Pro i’ve taken the following screen shot which shows the tables as data size 78 KiB. (not sure what the i means)

but just below it it says the Max Data size is 256 TiB (once again not sure what TiB means..)

Does this make any sense?

Thanks…

Hiya,

What you are seeing there is the sizes allowed for the entire table. TiB stands for Tebibyte and what you are seeing there is the maximum size for all the fields in that database.

If you take a look at the image I’ve attached this shows what the database field is set to which is TEXT which is only allowed to be 65,000 bytes (65KB) and I believe you may well be going over that size with everything that is stored in that table.

The only way I know of stopping this is to delete some of the IP addresses in that field so that it isn’t as big as it is at the moment I’m afraid, other than if the field can be changed to something other than TEXT in order to allow more information to be stored inside it.

One of the support staff will have to look into that though as I’m not really sure if that will work or not. I don’t think it would harm but best to get the information from the developers instead.

Best wishes,

Mark

Thanks Mark,

Will wait to see what Expression Engine support staff say…. Thank you

Kelli,

I can’t advise changing the type of field directly in the database. I’ll bring your question up to a Dev and see if there may be any repercussions. Setting the type to mediumtext would allow for a lot more room, but like Mark said, I’m unsure what might be affected.

Cheers,

Thanks Dan,

looking forward to your update

Kelli

HI Dan - we’re running into the same issue in 2.2.1.

Didn’t spot this thread earlier, but looks like we’re having pretty much the same issue: http://ellislab.com/forums/viewthread/197858/

(thanks to Mark for drawing our attention to this one)

Hi all,

I’ve chased this up to the dev team to see if we can get a recommended solution for this, even if it’s just a short-term fix. Obviously the long-term fix needs to be something more scalable than storing all IPs in a single database field, but that fix would likely take longer to implement. I’m gonna see if we can get something to you that’ll work well in the short-term. Hang tight, and I’ll report back when I have some more information for you.

Alright everyone, here’s the solution. In future releases of EE, we’re going to change the blacklist module to use a LONGTEXT field here rather than a TEXT field. In terms of scalability, a LONGTEXT field is 65536 times larger than a TEXT field.

In the meantime, you can run the following query on your database in order to change the field type right now. Only do this if you feel absolutely comfortable doing so and you already have a solid backup of your database.

Got it? Ok. Run this on your DB:

ALTER TABLE exp_blacklisted MODIFY blacklisted_value LONGTEXT DEFAULT '' NULL;

Report back and let me know if that takes care of the issues you’re running into.

Thanks Kevin - that’s working perfectly for us (or at least, blacklist is now accepting our additions and storing them in its textarea).  If for any reason the blacklist doesn’t actually act on that, I’ll let you know, but looks great to me so far.

Cheers!

Jon,

I’m glad to hear Kevin was able to get your issue squared away! If your blacklist gives you fits after this change, let us know by starting a new thread.

Cheers,