Thread

How do you think about this reccomendations?

CodeIgniter 2.0.2: Cross-Site Scripting (XSS) Fixes And Recommendations
http://blog.astrumfutura.com/2011/05/codeigniter-2-0-2-cross-site-scripting-xss-fixes-and-recommendations/

CodeIgniter is one of the most prominent “micro frameworks”, web application frameworks that prosper by offering their users unparalleled simplicity. It is unusual as a framework in that it does not make any reference to standard escaping mechanisms for views/templates such as the PHP htmlspecialchars() function anywhere in its source code, examples or documentation. This may create the unfortunate impression that users should instead filter input using an XSS filter function in the CI_Security class and do nothing on output. Users taking this approach may be particularly at risk from these security vulnerabilities.

My recommendation to the CodeIgniter developers, as documented in my original report, is to deprecate and remove the CI_Security class’ XSS filter. Responsible vendors should never persist in distributing and advocating the use of insecure software. I also urge them to revise their documentation to ensure that best security practice is noted in the area of writing views/templates and offer a shortcut function to an escaping mechanism for HTML output to standardise and ease its use by members.

Sanitise input, escape output. It’s the first thing they should teach at any web related course.

People might want to read There’s more to HTML escaping than &, <, >, and ”

As recomendation, thats should be apreciated. But if we talk about xss or any security issues here, at application layer level, i agree with n0xie that everyone which started their step into web developing environment must prepare theirself with good understanding about standard security procedure. We can’t, in one or another way, put this “security vulnerabilities” into any framework’s shoulder, and pointed them as a “Responsible vendors”.

6 words!

Never Never Never Trust User Input!

InsiteFX

Sanitise input, escape output. It’s the first thing they should teach at any web related course.

Quite true 😉. Unfortunately, PHP has an above average number of developers who are amateurs or self-taught who may not have the advantage of formal training. I’m largely self-taught myself (there were few web related courses back in the 90s and back then web security was not as well understood).

We can’t, in one or another way, put this “security vulnerabilities” into any framework’s shoulder, and pointed them as a “Responsible vendors”.

Bear in mind the vulnerabilities aren’t simply at the application layer. Yes, if you fail to escape the output then that is definitely YOUR fault and NOT the framework’s, but an XSS sanitiser is also used for HTML output you shouldn’t escape (e.g. RSS feed content is HTML and if you escape that it would not render as expected). In such cases, it’s not an application layer problem because the user has an expectation that the sanitiser works.

The core of my blog post comes down to one main thing: clarifying the documentation. As much as we can refer to escaping being essential in a forum, the fact is that it’s not mentioned anywhere in the official documentation or the CodeIgniter source code. It would be a very minor task to add this, clarify when and where a XSS sanitiser should be used, and how to properly escape output. Adding an escaping function would likewise be a minor task.

Accurately relaying security vulnerabilities pales in comparison to encouraging users to use better security practices.

Bear in mind the vulnerabilities aren’t simply at the application layer. Yes, if you fail to escape the output then that is definitely YOUR fault and NOT the framework’s, but an XSS sanitiser is also used for HTML output you shouldn’t escape (e.g. RSS feed content is HTML and if you escape that it would not render as expected). In such cases, it’s not an application layer problem because the user has an expectation that the sanitiser works.

A data integrity policy is a set of rules and regulations related with how your application ensures that it stores and outputs expected data. It cover everything from checking that data is valid within your character set and encoding, through filtering undesirable characters, to contextual integrity, such as ensuring the well-formed-ness of stored markup. At least we can chop this concept into two main approach :

First, a workable data integrity policy is based on the founding principle that data inside the application is pristine. That is to say, incoming data is filtered at the border and stored filtered. Outputing the data can then happen with no proccesing required. This approach is sensible for two important reasons. First, filtering is not a trivial efforts, especially for encoding conversions and syntax checking. Second, a typical chunk of application data will have fewer input than output vector. Outputing can then become as easy as reading stright from the data(base) and echoing the content, safe in the knowledge that the contents is valid.

And actually, you were just talk to one approach regarding with data integrity policy concept, the second one.  Of course, not all filtering should necessarily be performed up front, though. When the data we talked is in an XML document, it will need the four XML meta-chars (<, >, ” and &) to be escaped. But when , lets say the name field, is displayed in an email or in javascript, it doesn’t need to be escaped (or needs to be escaped differently).

So, should we store the data escaped and unescape it for the edge case, or store it unescaped and escape it for typical case? This is very very (much) a point of style. I prefer to store all data unescape and escape it during display, but reserve of that can easily be argued by someone else (like n0xie approachment for example). Set a standard for your application and stick to it - mixing the two methods is a recipe for disaster, leading to unescaped vulnerabilities or double-escaped uglyness. The policy we came up with, should reflect the way we use the data within our system.

Also, as i stated above, your whole article just covering security/vulnerabilities issue on application layer, furthermore, the outer of application layer! You didnt said anything else, related security/vulnerabilities issues on other layer, for example : multiple disk in mirrored or parity RAID configuration help avoid data loss, or about backup tapes and offsite backup cycles aid in disaster recovery and so on.

At the end, look again at my previous post, and you will find that I think, introductory articles on security issues such as yours, must be apreciated. The only objections have come from me, how can you have a conclusion, later, which connects the “security vulnerabilities” with “CodeIgniter developers” as “Responsible vendors” ? It doesnt make any sense for me. Its really like a mechanic who blame the hammer and screwdriver (his own equipment) when the door that he made were poor and resulted in his house robbed.

At the end, look again at my previous post, and you will find that I think, introductory articles on security issues such as yours, must be apreciated. The only objections have come from me, how can you have a conclusion, later, which connects the “security vulnerabilities” with “CodeIgniter developers” as “Responsible vendors” ? It doesnt make any sense for me. Its really like a mechanic who blame the hammer and screwdriver (his own equipment) when the door that he made were poor and resulted in his house robbed.

I’m not sure how you’re interpreting my blog post so I’ll clarify. My point was simply that distributing an XSS sanitiser which, in all probability, has undiscovered security vulnerabilities is not the mark of a responsible vendor (in this case, EllisLab and/or the CodeIgniter developers responsible for this function). Bear in mind my security report specifically recommended deprecating and removing the XSS sanitiser due to its high risk of being compromised. Let’s be brutally honest here that EllisLab is not alone. PEAR, for example, continues to distribute a package called HTML_Safe which I also recommmended be removed - and it is an entire order of magnitude worse than CodeIgniter’s XSS sanitiser. Would you call PEAR responsible?

It is simply my opinion that shipping risky software and downplaying the security impact of reported vulnerabilities (whether though innocent omission or not) is an irresponsible act. I’m trying to help here. I committed time to reviewing CodeIgniter at no cost and, despite having no reply from a developer, still kept my mouth shut for six weeks giving them all the time in the world to take action. The only reason I even blogged it is because of the lack of accurate disclosure.

As a user you should be asking EllisLab why some guy could report 7-8 vulnerabilities in the first place and how they will ensure such a security scenario will be actively prevented going forward. Hoping something is secure does not make it so, and the next review may not be performed by someone as friendly 😊.

*Sigh, i think we were talked about security/vulnerabilities issues here. I was give, at least general understanding of data integrity policy, at my previous post (and again, i didnt see you quote any of them, since you’d always just quoting, the most un-necessary part of my posts.).

I think it will be better, if before anyone read what you called “security report”, they understand what i mention above. Or, i missed the point of your article? Or, your point at that article, is just complaining some lack of “Open-Source” framework’s disclosure instead talked about security/vulnerabilities issues?

Padraic, thank you for your comments, and your blog article.

The disclosure of the incident is not enough, I feel. Official Information about the security bug is too little.

And the CI documentaion about XSS protection is not good, at least the best security practice http://ellislab.com/codeigniter/user-guide/general/security.html is not the best.

I think that improving CI documentaion benefits all CI users.

*Sigh, i think we were talked about security/vulnerabilities issues here. I was give, at least general understanding of data integrity policy, at my previous post (and again, i didnt see you quote any of them, since you’d always just quoting, the most un-necessary part of my posts.).

I think it will be better, if before anyone read what you called “security report”, they understand what i mention above. Or, i missed the point of your article? Or, your point at that article, is just complaining some lack of “Open-Source” framework’s disclosure instead talked about security/vulnerabilities issues?

The only point of my article was to disclose the fact that vulnerabilities existed in prior version, that CodeIgniter 2.0.2 fixes most of them, that improving the documentation would be a good thing to resolve any remaining uncertainty, and that there were a couple of simple things anyone using CI could do to improve their security in that specific area without it costing an arm and a leg 😉.

There are, of course, lots of related security practices we could rely on. And I agree that consistency is important. Just as important is convention, which for the sake of promoting simplicity in PHP, has always leaned towards escaping on output (easier to audit, PHP has fast functions for it, it’s near foolproof barring a PHP bug when used properly, etc.). Makes more sense than relying on an XSS sanitiser that might be vulnerable (not just picking on CI - ALL sanitisers will eventually have a vulnerability reported including HTMLPurifier every year or so).

Padraic, thank you for your comments, and your blog article.

The disclosure of the incident is not enough, I feel. Official Information about the security bug is too little.

And the CI documentaion about XSS protection is not good, at least the best security practice http://ellislab.com/codeigniter/user-guide/general/security.html is not the best.

I think that improving CI documentaion benefits all CI users.

I agree, and updating the documentation would take very little effort for potentially a lot of gain in educating users on how to write more secure apps with CI. That can only benefit CI’s already positive reputation.

I really dont know, your impression, when write that article. If you give yourself more time, to understand what I said, before hastily responded in part that only one part of the core that I wanted to say, then you will realize that there is no single thing about security techniques, which I rejected, of your conclusion that emerged from your research about the XSS attack. And of course, this needs to be appreciated.

In fact, I’m just trying to give a more thorough understanding, related with data integrity policy, as the basis (once a complement of what you wrote in your blog) when people start talking about XSS attacks. If not, then for me, same as you invite people to discuss an exciting game of football, without giving the basics of understanding of the rules in football (which most of them, may not know about football)

I am here not to talk or want to act like a lawyer who defended the EllisLab. It’s just a little funny, seeing the emphasis on writing is more towards the demands that they must include all points of your “security research” details, rather than promote what’s developers need to know when they interact / manage their data integrity policy, directly . And as I tell you, and this is purely my opinion, that such childish demands, more like a mechanic who blames his tools (and the sales also stores that sell these tools, even in fact they sell it for free.)

Has it occurred to you that I didn’t quote the first part of your response because I agreed with it? 😛 I have no dispute with what you said. What I did quote was something you said made no sense to you, so I added a clarification in the event there were any misunderstandings. I didn’t want to go too far off course from the original topic.

In the same way, I’ll clarify that my “demands” are simple recommendations. I’m not holding a gun to anyone’s head - just offering the benefit of some 14 years of PHP and security experience for consideration.

I apologise if my references to a “security report” are a bit vague. I don’t wish to disclose the full details of the vulnerabilities or the original report given EllisLab has stated in the past they prefer not to, and I want to respect their wishes in this regard. Hence my article doesn’t carry the usual detail I tend to disclose and limits itself to a quick overview of the count and generic nature of what was reported.

Interesting discussion… I’ve recently completed a 1.5yr Diploma IT (Website Development) course (note: not a full Bachelor Degree of Computer Science, where I imagine things would have been taught more in depth. And to further compact the problems I’m about to mention, my course was initially a 2 year course, but my institution condensed it into 1.5 yrs…hmmphfff).

1.5 years is not much when you consider you have to cover topics such as HTML, JavaScript, PHP, a FrameWork, Relational Databasing, (My)SQL, UML, Project Management (and an array of other ‘interpersonal, business, costing, budgets, Service Level Agreements, etc’ kind of subjects), Online Privacy, Risk Analysis, Reporting/Analysis tools (ie google analytics), logs, Testing and so on. And that’s not even to mention stuff like a JavaScript Framework/Library, SEO/SEM, .htaccess, JSON (etc etc) were not even touched upon. As such, unfortunately, Security (like the rest of the subjects we studied) was skimmed across in a very quick fashion.

As such, I’m going to say that both ‘opinions/contributions’ made by Toopay and Padraic Brady are of utmost importance and interest too me. Trying to figure out ‘what constitutes good security’ in todays day and age is almost impossible, there are ranging conflicts of opinions and methods on every blog you read.

Having a good solid “data integrity policy” is imperative, but at the same time I fully agree that the documentation on CI is rather lacking, especially for someone, like myself, who is relatively new to the game… For example, I had to ask on the forums here about CSRF - the documentation is only 3 lines long.

A good tutorial on how to safely and accurately implement the myriad of security goodies CI has given us, combined with an overview of ‘why’ it is important to implement them in the first place, (and perhaps even covering how(and WHY) to write your own custom form filters and extend the base validation class, thus tackling the filtering issue at its base) I believe, would be up great benefit to this community. I know I would definitely be watching/reading it!

I shudder to think what kind of applications some ‘novices/hacks’ is making using CI (or any framework or just a plain old text editor for that matter) considering how hard I’m finding the whole ‘security, filtering, cleansing, input, output’ minefeild. Whilst I understand it IS NOT CI’S RESPONSIBILITY OR PROBLEM, as mentioned, a good tutorial and improved documentation would be invaluable… then when ‘novice’ questions arise in the forums we could just point them in the right direction so that they can build better and safer applications, and thus leaving the forum for the more important and technical aspects of this complex and daunting issue.

Just my 2 cents worth from the perspective of someone struggling, but trying hard, to build safe applications for my beloved users.

Security is always a complex topic. You’ll probably find when it comes to programming languages specifically that we focus largely on the practical elements of what to do, when to do it, and how to do it - and less on WHY to do it. The WHY is actually a giant minefield. I know very knowledgeable programmers who when confronted with WHY are at a loss to explain it.

Take a simple example (well, relatively simple 😉). If you use Google Code Search, you can find examples of tons of PHP classes and libraries connecting to HTTPS URLs (Twitter’s API for example). Now this seems a simple operation! Boot up curl and away you go. If you look really hard though, you’ll notice that many clients pass in a curl config to disable something called Peer Verification (SSL_VERIFY_PEER if I recall the option name correctly). This option is explained almost nowhere in depth for PHP. In fact, PHP Streams disable something like this by default (just as Curl enables it by default). What does it do? It verifies the SSL certificate of the server you contact - i.e. it verifies you really are talking to Twitter, and not some hacker with a fake SSL cert who’s interested in your Twitter API session 😉. So, if you disable SSL Peer Verification, it’s an insecure connection. Yet, it’s commonly done.

Try explaining the WHY of that to most developers, and their eyes will glaze over 😉. Many web developers probably don’t even know exactly how SSL works to even see that it is a problem. Often, it’s just easier to document it very quickly (in a short paragraph as above) and then basically give them a good rule to follow (NEVER DISABLE SSL_VERIFY PEER!).

I still know some developers I admire who struggle with this 😉. You can imagine how complex it can get as the range of security topics increases (as it always does, look up remote timing vulnerabilities that I blogged about recently for an example of who the WHY is so damn complex - it’s a long post 😊).