Post Pen-Test scenarios for the CISO

A Penetration Test can be hard for a company. Over the past years when I have been doing Penetration testing for companies, a lot of them undergoing a pen-test for the first time in their corporate existence. The results are anything but cheery. It is usually a gruesome mess of unpatched servers, misconfigured network devices and wildly non-secure apps that can weaken the resolve of the steeliest of CISOs (Chief Information Security Officers) or their equivalent.

A typical scenario emerges. The CISO receives this 'bloodstained' Pen-test report from the pen-testing company and one of the following usually happens:

• The CISO utters words like 'This problem will be fixed in the next week, or I will not rest' and subsequently files it in his cupboard and forgets about it for posterity until there is a massive security breach.
• The CISO delegates to the IT team or the team handling the implementation and they provide an excel sheet saying that they fixed (even though they didn't) and the CISO believes it because they are (supposed to be) doing their job. The problem festers.....
• The CISO receives the report and shares very limited (or even wrong) information to his key staff because of 'security reasons' or reasons of distrust. This leads to limited action because no one trusts anyone and the organization stays blissfully insecure (and not only emotionally).
• The CISO doesnt understand the technicalities. Blanket statements about IT security and governance are made and the ground realities are forgotten until a security breach shakes the very ground beneath them.

A successful pen-test is the first step towards better Information Security for an organization. The focus of a CISO should be to leverage on the results of a pen-test and drive efforts to ensure tactical information security and strategic security directives for the organization. Here are a few focus areas for the CISO who wants to get the best from a pen-test:

Quality: A good quality pen-test should be the first focus area for the CISO. The pen-test is conducted against the scoped IT environment (the IT components in scope for the test). The pen-tester should have used a solid methodology (Look for a specific 'methodology' section in the report). Most Pen-tests are not really Pen-Tests but cursory Vulnerability Assessments where the pen-tester has run some automated tools and identified vulnerabilities. The focus should be on depth, and depth is achieved through penetration attempts against the target IT component and the results from said penetration. A critical aspect of a quality pen-test is the report. The report has to be clear, comprehensive and provide specific recommendations on the given vulnerabilities. The report should ensure that the implementers should be able to comprehend and implement the provided recommendation. Another note to CISOs, DO NOT automatically accept a lower cost penetration test, it usually means lower quality. These are matters of your organization's information security.

Communication: I recently came across a CISO who wouldn't share results of the pen-test with the Implementation teams that required to fix it, citing 'Security reasons' as he didnt trust any of them to deliver. Many CISOs do not follow the 'Trust but verify' rule because they do not have the skills to verify. They provide limited (or even wrong) information to their implementation team on fixes and these cursory fixes are of little value in effectively correcting the vulnerability in the system(s). CISOs should communicate effectively with the Implementation teams, providing them enough information required to comprehensively fix the issue. Meaningful data, such as screenshots, downloaded files provided as a part of the pen-tester's results should be provided to the implementation team for them to grasp the issue and fix the vulnerability as effectively as possible.

Project Management: A Penetration Test, from a CISO's standpoint is a project in itself. The CISO has to define a project management plan for the fixes of the vulnerabilities. Based on the Risk Ranking of the vulnerabilities, the fixes should happen based on defined timelines. The culture of Information Security is tough to implement, as people will naturally tend to be convenience oriented, as opposed to security oriented, and this behaviour would manifest itself as vulnerabilities in the IT systems of the organization. The CISO has to cultivate a culture of security execution, where every team responsible for the fixes, delivers on the fixes and these fixes are verified by effectiveness and propriety before being signed off on. Sometimes, there are certain long-term or deep-rooted fixes that take a much longer time or effort in fixing. For instance, implementing encryption on a production database containing millions of records. The implementation of the fix in this scenario is a complicated one, requiring redesign and downtime. In that case, the CISO should actively consider and design compensatory controls to ensure that the lack of primary control is suitably compensated with the secondary control. The CISO should also contract with the pen-tester (either internal or external) to perform a re-test of the previous scope to ensure that the vulnerabilities have been fixed.

Review of Processes: Oftentimes, I have noticed that CISOs blindly fix results without paying heed to a flawed process/culture that would have led to the vulnerability existing in the first place. This is commonly seen with patch management, where a flawed patch management process would lead to inconsistent application of security patches across critical systems, allowing vulnerabilities in previous versions to still be at large, providing easy access to an attacker through a Code Execution exploit or a Denial of Service Exploit against the vulnerable system. The CISO should review and relook at the processes and procedures that exist in the organization and consider amending/overhauling the processes based on evolving security threats and the organization's response to it. This can ideally be achieved by reviewing pen-test or Vulnerability Assessment results from previous quarters or years.

Consistency: A Penetration Test is not a one-time activity (or atleast, it shouldn't be). The CISO should ensure that a pen-test is conducted bi-annually with a quarterly Vulnerability Assessment. Threats evolve consistently and exploit code is written every day for myriad software and applications. This necessitates the need for a repetitive assessment of the organization's IT environment over a period of time. In case the organization's IT environment is massive. The Penetration Test should cover representative samples of all IT components like routers, firewalls, desktops, servers and applications. The findings from all these tests should be used to harden the rest of the components in those sample classes.

Penetration Testing is a tough gig to take for entities. More often than not, they find themselves staring at very adverse results. They often lose heart and this exercise becomes nothing more than lip-service or a compliance check. Using some of these techniques, I have discussed (and I am sure there are many other concepts) I think CISOs or their equivalents in organizations can make a positive and meaningful change in the security stance of their organization.

What's wrong with Penetration Tests, and how we can set it right (India Edition)

Penetration Testing is complicated, especially so for organizations that have to fix the issues from the debris, that is their IT infrastructure components. Over the past few months, I have had tons of experience leading and handling pen-tests for companies in the sub-continent and decided to rant. I thought of writing some of the problems that are out there and also some possible solutions to some of these issues, which will make these pen-tests a whole lot easier and a whole lot more efficient. They are:
Test Everything - Fix Nothing
This is a condition I have seen management typically have with Internal Pen-Tests or Large Application Pen-Tests. Management would like to include a ridiculously large scope of components to test. They include everything from their Database server, right from the laptops that they use at home as part of the Pen-testing activity. The results in most cases (especially with Internal Pen-Tests involving client side systems) is really ugly. Multiple exploits, backdoors and in some cases, traces of popular worms like Conficker (yes, I am not kidding). What follows is a gigantic report and what follows that is......Nothing. I have often seen that organizations who adopt this policy usually dont get anywhere in fixing the problem. They find so many loose ends to tie up, they huff and puff and eventually pack up and go home. This brings me to my first point in this rant-post:
Start small (or manageable) - Many organizations (especially ones new to VAs and Pen-tests) usually go gung-ho and then fizzle out after seeing adverse reports. My advice to you is to test everything in doses that you can handle. Prioritize on the critical components first and then phase your testing for across the year. Also, testing everything (literally) may not be required. Mirroring results for similar IT components/applications is easier than literally testing every single component. For instance, upon finding security holes in a Debian Server, it is probably a good idea to fix issues on a sample of servers and roll out similar operations across the other similar servers in the environment. Additionally creating a solid Hardening standard for Debian Servers, coupled with Patching would not necessitate the need for testing every single one of these similar components.
Ridiculous Time Frames
Sometimes, we are given ridiculous time frames to work with. "Hey, can you test my E-Commerce app in two days. I dont have more time than that. Also, I have to fix the problems after that." and my reaction to that is usually "Oh, you wont have to worry about too much to fix, I probably would need two days to just understand your site, and since you only have two days, I will give you a clean report" in my most sarcastic (and borderline mocking) voice. While time is a constraint, such super constrained timelines only result in a untested and potentially non-secure environment. I am sure no one, either management or the pen-tester would like to turn up with false negatives and find that their application/server/network component was super-vulnerable only because they hadnt the time to possibly test extensively. That will come back to bite a lot of people.
Fixers are Breakers
This is a condition I normally see with Application Pen-Tests, where the management is extremely clued in on the test before the commencement. Statements like "We would like you to be extremely comprehensive and give us all our security holes right between the eyes". Later when we deliver our report of their Hindenburg-like app and discuss with them, the very same people become the worst enemies of everything sane and secure. I would be discussing a gaping business logic flaw about a failed authorization flaw where a user would get to play admin with the application and they try to find explanations which are on the lines of "but the user would not be able to do much even if he/she became admin" or "I think this was an intentional feature that we had in case this eventuality." Earlier, I used to have a small explosion in my head, but yoga has taught me to react like Mr. Wolf from Pulp Fiction "I am here to help, if my help's not appreciated, tough luck gentlemen"

Why attackers love your developers!

Kind Sir

My name is Developer A. I am having a problem with database connectivity to Oracle. I think I am doing everything right, but I am getting an HTTP 500 error with the following details:
(Full Stack trace follows)
The sample of the source code is given here.
(Source Code follows)
The error where it is occurring is at this URL (Organization's url follows)
Please help me out with this as soon as possible. I reeeallly need help!!!
Email me at developerA@organization.com

Yes. This was one of the messages I found sifting through my Google hacking results while I was pen-testing a web application the other day. Needless to say that, the page with the stack trace was open for me to go and gain a complete understanding of the application and penetrate. The organization which I was pentesting had gone great distances to get secure. They had spent a lot of time, money and resources in getting secure and staying that way. They had gotten a few things wrong, but mostly they were on target. Unfortunately, they had forgotten to explain to their developers that dirty laundry (or in this case confidential laundry) should not be washed in public. I have seen this with several organizations (mostly in Software development) where developers post their queries in Internet forums and get some other professionals to look at it and give them some insight into the matter. While there is nothing wrong with this practice, the way it is followed is quite shocking. Developers dont even go through the trouble of hiding the name of the organization they are working for. They advertise their email address and in several cases the URL of the page which is problematic and buggy. When posting code snippets or source code, they mostly never remove sensitive details like certain class references or database drivers (sometimes, even usernames and passwords to dbs) and last but not the least, they use words like "Kind Sir" on an Internet forum (although the last one is not a vulnerability, it is bloody irritating). I am sure all of you would realize that such information can be extremely useful to anyone looking to break into an application or a site. It would be worth its weight in gold. And, of course, as usual the only thing you really have to do is "Google it" or "Bing it" (my vendor agnostic comment for the day).

My advice to organizations to prevent against such ignominious disclosures are these:

• Instruct developers never to post in internet forums under their own name or with their company credentials like emails, etc.
• Encourage developers to first find help within the organization. I have seen that several greenhorns tend to be afraid of asking their seniors or project managers any doubts regarding the code, fearing a nasty remark or backlash. Build an environment of openness and encourage questions. Not doing so, would result in these juniors asking questions anyway, but to someone you are totally not aware of, not bound by confidentiality agreements, etc.
• If someone really has to post queries on the Internet, make sure that these are approved. Source code should not be posted. Stack trace should not be posted. Questions should be based on peripheral details and no sensitive information should be posted. Words like "Kind sir" should definitely be filtered out of posts ;)
  
• Have your risk folks scour the internet regularly checking for violations and take action against people violating these instructions.

Google Hacking with Webex - Cool Recon tool

The idea for this actually sprang from a webex conference I was involved in recently. Before what turned out to be an eventually boring conference, with esoteric Powerpoint slides being shoved into my screen. I received an email from the organization that was hosting the meeting. Upon clicking the link to join the meeting, I was directed to the organization's WebEx Enterprise Site page. where I signed in using a Conference ID and a meeting password. Simple enough.

After the odious meeting I attended, as I signed out of the conference, a feedback form was thrown at me asking me how the meeting went, which I duly ignored and went about my business. This idea actually sprung in my head when I was sitting through the conference and thought about using WebEx to perform some initial reconnaissance activity for a Pen test. I noticed, before signing in that the organization had planned some more sales meetings with what looked like their prospective customers, which intrigued me.

I then put my Google Hacking Hat on and wrote a simple Google query intitle:"WebEx Enterprise Site" based on the title I had seen in the organization's WebEx site. Not surprisingly, several organization's WebEx links popped up in the Google Search results. As I started exploring around, I noticed some very confidential information up for grabs in several sites. Not only could I gain some valuable information about some sales meetings and internal group meetings they had lined up with their prospective clients or already-existing clients, not to mention internal organization members, but also some of them had recorded their conferences and these recordings were available for me to view.

While this is not a "Security Vulnerability" in the traditional, technological sense. It is a way for competitors to get some valuable info about your potential clients or your new offerings to the market (some internal meetings had the names of the products or projects these organizations were currently working on). This, is also an excellent way to perform an all-round Penetration Tests. While the ordinary Pen-test involves performing garden variety Network and App layer exercises, these methods can add significant value to a pen test and provide a bit of the old "Out of the Box" information.

Incredulous Interactions with Web Application Developers: 2 of Many

Here's wishing that all of you have a good weekend. I left early from the office today to attend to some household duties. While I was sitting and watching TV, I came up with a concept, which resulted in this title. I interact with several application developers over the past few years and now as I steeped in AppSec, I have been interacting with a lot of Web Application Developers during my application assessments. Normally, they are a nice bunch, who are quite convinced about the authenticity of my claims of having their applications torn apart on the internet. But, everytime I discuss security, there are always this few developers who end up astounding me or completely disarming my constant efforts to bring about change in the way they think (or don't think) about Web Application Security . I thought, "Why not chronicle my interactions with these people, for the benefit of the community, and for a few laughs (or atleast a smile) in the normally tense lives of security folk?", and so, I present to you, the second in the series of "Incredulous interactions with Web Application Developers". You might be wondering what happened to the first of the series. It is the first entry in this blog, but has not been grouped under this title. Anyway here it goes....

It was at the end of the day, and I had almost finished assessing this application. The application was being developed by a small group of developers working for a small company. Their product claimed to be the only one of its kind and one of their potential clients wanted the application to reviewed for security.

I had reviewed the Application for over 2 days. I had spent some time interacting with the project manager and the developers, and had performed a combination of black and white-box tests on the application. The application had some serious issues, there was almost no validation and whatever little validation it had, it was on the client-side, utilizing trusty old Javascript. I knew it was a problem. The developers were quite sold when I showed them a Cross Site scripting PoC as well as a Embedded Flash injection PoC. The only problem was getting through to the Project Manager. He was not convinced, and a conversation on the following lines ensued.

I used Burp proxy to perform my scripting attacks and he said this. "This application is going to be used only by the Company employees. why will they want to steal this data from the company?", for which I gave him standard insider threat lecture, statistics, et all. I tried explaining to him the fact, that this application stored extremely sensitive data (which it did) and an insider would also have a clear motivation to break into the application to steal information, not to mention disgruntled employees, or other employees with malicious intent. I also explained to him the CSRF angle, where this site might be attacked by another site violating the same origin policy.

He was yet unconvinced and he said, trying desperately to prove a point, "I have worked for over 12 years in Software development and have developed dozens of applications, never before has a client asked me for all the security features you are asking me for, it seems that you just want to prove a point." I burst out laughing at this and told him that if he believed that his potential clients weren't concerned about security, then they wouldn't have engaged my services to perform the assessment. I also had to tell him, that the way he or anyone else has been developing web apps in the cavalier manner that they have been all these years, does not make it right. Quantity is not quality, I told him and I also informed him of the fact, that awareness has been rising in the community about the effects of web application attacks, which is the reason why, if he wants his product to remain competitive and viable in any market, he would have to do "complicated things" like input validation based on a whitelist of "known good" characters to prevent against a smorgasbord of attacks. I told him finally that it would be for the benefit of the application and the data it holds as all these measures would prevent against any untoward incidents and not result in loss of reputation for his company. The man finally backed down after that and we discussed some implementation strategies for input validation across his web app.

This got me thinking, that developers have churned out several thousands of web applications which are working all over the internet and organizational intranets over the years. This was a single project manager from a single small company, who had himself been involved in the development of over 20 applications in his career. It would be quite scary to think of other developers over the years who would have developed similar applications with similar if not worse vulnerabilities.........

Overreliance on Javascript: Pen-tester's pet

I am sure all of you have been acquainted with the input validation situation where the developer validates all his input on the client-side using some poorly written Javascript to boot. Well, I recently experienced a new high in Javascript overreliance, and this was precious!!

I came across an application which relied heavily on Role Based Access Control. The Application is an Enterprise Application which has several types of users and several different "screens" that each user can access, thereby restricting the user's access based on the user's role. Well, this is how a traditional access control system works.

So, what would one do to ordinarily circumvent these controls? The first thing I tried was the URL route, where I scanned the app for URLs and entered these URLs to elevate my privileges. I was easily able to do this and the pages that clearly weren't meant for my user role kept appearing. When I tried to input some data and submit the form, the server threw me a big fat Javascript alert, "Access Denied".

Then I revoked the temporary permissions on my NoScript and the entire application fell apart. The Application so heavily relied on Javascript that the access to the links was controlled using Javascript only.

The code went something like this:

function Menu()
 {
 var UserRole=document.getElementById('userRole').value;

 switch(UserRole)
 {
 case 'elevatedPrivUser':      
  document.getElementById('thisScreen').style.display='none'; 
  break;
 case  'lowerPrivUser':       
  document.getElementById('someOtherScreen').style.display='none';
  break;
 case 'evenLowerPrivUser':
  document.getElementById('yetAnotherScreen').style.display='none';
  break;

As you can see here the Developer had left this brilliant bit of Javascript on each and every page. This was the basis of the Role Based Access Control System of the Application. Then I thought, surely, even if I accessed any one of these pages and tried to submit a request, the server would stonewall me and throw me out. I tried it out. Guess what?....More Javascript. Luckily, the Javascript wasnt available on plain view in that page, but clearly Javascript was the only validation which had been provided to validate user role and by disabling Javascript, I was able to pass my request over to the server which was accepted instantaneously.

So, the next time I go try to test an app for elevation of privileges, I will not bother with the session hijacking or the sniffing. I will just ask my NoScript to do the job and I will check whether the developer believes that "Client-side Scripting is the best!!!!"

Payment Application Pacman

Lately, I have been flooded with Application Security Assessment work. Applications of all forms, shapes and sizes have been given to me to test, which I have gleefully accepted. The work is supremely interesting. I always knew that there were tons of applications being developed all over the place, but I always figured that they would be of a certain nature, like a plain vanilla E-Commerce Application or a Banking Application, etc. Because of the spate of application security testing I have been doing recently, I have found that there is an application for everything. Right from Flea-Market Management to Cold Storage Management Applications. The granularity of business verticals which applications are catering to today are mind-boggling. Wait, I think I have diverted from my story, haven't I? Damn, I do this all the time, okay, back to the story.

I was testing this application for Payment Card Industry Compliance recently. I was quite impressed with the application at the outset. The application seemed to be loaded with some innovative features and it seemed like a great application for the business vertical it was hoping to address.

I started to test the application a few days later. It was sometime in the evening. I was not in the mood for any more testing. I had done quite a bit that day. I had already finished a preliminary vulnerability assessment of the server and as the assessment would be performed over a few days (yes, I procrastinate, a lot!), I thought of playing some Pacman. It is one of my favourite games. I thought of a slightly weird way to play it. The application I was testing was structured in a way where the user would use the input fields to enter values to be stored in the Database and as the Database updated, the previous inputs would be displayed in the table below it. Great way to test for persistent XSS, thought I. Therefore, I entered this harmless reflected XSS script alert in a field


and this is what I got


"Not bad", thought I, "the developers have managed to keep the bad apples out, or could it be the simplistic process of weeding out the angular bracket using a even simpler client side validation routine?"

Not to be deterred, I fired up my ever-trusty WebScarab and decided to take this attempt at killing two birds with one stone (namely me wanting to play Pacman in a different way and app sec testing) and caught a perfectly harmless request and gave it a interesting twist. I entered as input an html embed pointing to the Pacman Flash game, a lo I was gleefully playing Pacman on an application which had no business storing this in its Databases. One part of my test was done, quite a finding too. Cross Site Scripting (and later CSRF) vulnerabilities rampant on this site.

Coming back to my initial tangent. There are around 108 million distinct websites in the world today and even now, 23% of the world's population is exposed to the Internet, which is looking at a meteoric growth pattern in the future. According to the WASC (Web Application Security Consortium), almost 50% of web applications are vulnerable to Scripting and Information leakage vulnerabilities. As development becomes easier for everyone, we are only going to see more number of sites and apps falling prey to session hijacking, SQL Injection, Denial of Service and many other evolving attacks.

BTW, You can play the evergreen Pacman here