.NET Brisbane, Australia

Issues between HTTPS, IIS and Sharepoint

2009-03-08T15:49:00Z

Hi guys, I want to share with you a discussion that I’ve participated recently.

Consider this: you are a a host provider and your company will offer Sharepoint support to the public so they can pay you a monthly fee and then they setup a Sharepoint site with you.

You go for a simplistic and cheap design. You have IIS, Sharepoint installed, and you create a web application on the port 80. Within this web app you create multiple site collections. They are the sites your clients will have control for their own setups.

And how do you host multiple websites on the port 80 with a single IP address? Easy. I will use host headers – you say.

And you think: I should be fine. I will setup in away so each client will have their own separate database, they will redirect to the given URL I will provide them and according to the hosting plan I enable disk quotas for each case. For instance, if my client pays me some little money and he is a ’silver client’ I offer them 10MB; if they are ‘platinum client’ and pay me some more I give them 50MB.

All goes fine and well. You see everybody joining your company, the clients keep coming and your setup is totally independent.

so, what’s the problem here?

You are unable to offer HTTPS in that setup. If one of your clients wants to add a shopping cart area and want it to be secure, you can’t help them with that model.

The thing is, IIS can not resolve an incoming HTTPS request like that. ISS will hold the request and it is going to ask himself: ok, to which website should I give this request? Unfortunately IIS as of now can not address this question following that setup. Everybody is under the same IP.

To fix that one of the recommended approaches is to follow the diagram below:

On your IIS you will create multiple web applications, then for each web application you will want to give separate IPs and then host sites on these IPs, and then these web applications are the ones you will offer to your clients.

That’s a little bit more elaborated solution, a little bit more expensive but the gains in scalability will overcome the headaches you might have in the future with an atomic structure that at some point needs to be flexible.

And on that model IIS can finally then fix the SSL dilemma. Because then you will assign the applications to independent IP addresses on the ISS Manager.

Sometimes it takes more than a quick and simplistic approach to design a solution if you only know partially the products involved. Once you have the proposed design please be aware of the issues that are basic in the parts involved. In our case, a knowledge of IIS would have avoided a big trouble in the beginning.

See you later.

Filling the SOA gap

2009-03-04T13:35:00Z

Hi guys,

Let's talk architecture again, reference architecture, more

specifically SOA, and how can we map the available products and resources available from Microsoft given a SOA project?

Before I start, let me make a statement here: I will talk this from the Microsoft's point of view, since this is a blog about Microsoft technologies.

In any case, it doesn't matter what's your preferred provider as long as you are able to correctly do the mapping of functionalities to better fit your business' plans, budget and SLA. And just to revisit: SOA is an architecture where the functionalities of existing business applications are exposed and published as services.

And what would be a service? Services are software components that expose application functionalities in a given SOA architecture and they are:

- self-manageable;
- message-based oriented;
- can handle and support many protocols;
- can be published on a myriad of hosts;
- implement operational contracts, interfaces and message types;

Of course you can design some service that doesn't follow these rules, but let me tell you: Rules are made with a purpose and in our case the purpose is to design a solution where clients and services are highly decoupled, thus paving the way for the reuse of functionalities. One of the goals is to maximize the resource utilization in our project.

I won't talk here about governance, granularity, message routing, service level control etc. I won't go there for it is a much larger topic, so this post is about the technical view. Now, let's make our first diagram given what we've seen so far:

As we can see, we have here the common services of a SOA architecture such as the presentation services, collaboration, systems integration, orchestration services etc.

Looking at this diagram we can identify the aspects that we really want to map in our solution to be successful. Note that in some scenarios sometimes the security is critical, sometimes the orchestration is paramount, sometimes the platform integration is more important.

Now, visualize this filling the gaps with the products that Microsoft has to offer.

An interesting conclusion we can take at first sight is that some products can cross domain frontiers and can handle various capacities at once, such as the Windows Workflow Foundation, which can be used for interoperability services and orchestration at the same time with BizTalk Server.

What's the best option to choose? Well, to be able to see this big picture and to choose the best piece in the puzzle is your job as architect. Unfortunately many architects fall in the problem to overkill the solution.

Also important is to choose not to overkill the solution. Sometimes a simple custom application can fill the gap enough to not require a bigger solution like Windows Workflow Foundation, for example. Otherwise the whole project just becomes harder to handle and to maintain...thus increasing the ROI overtime...and the developers patience.

The message to keep in mind: any good architecture is composed by many capacities. To identify these capacities and which one of them are important for our solution is as critical as choosing the technology provider.

See you later.

The perfect entity model development framework

2008-08-06T06:08:00Z

The Entity Framework is out there and it is common term already in the framework talks. There are hundreds of sites dedicated to its best applications and best architectures tactics.

When an architect is involved in these talks the good ones always bring the discussion up to a helicopter view, since the discussion will inevitably move towards the best model and to comparisons against the 'what-would-be-the-perfect'.

Now seriously, is there any perfect framework model?

The purists’ attacks against the Microsoft EF always say that "EF is not an independent layer neither multiplatform oriented", so it can't be reused and integrated with ease across the other systems in the enterprise, for that would be the dreams of any architects.

Since the introduction of shared folders you could easily for example place a text file in some URI and use it shared across many systems. Obviously as we can see the data would not be cached and few few mechanisms were available to raise a notification to the data holders about that update. The EF is not supposed to accomplish that task as it is a framework to access data using objects.

So here it comes another point: should we put a data access layer above it?

Remember, Microsoft data access methodologies have been changing dramatically over the last 5 years, which paves the way and gives us hints that in a near future it will change again. Legacy systems will be always a reality of the high paced IT market. The arguments against it is that having so many layers in an application will over-engineer the problem and will affect the performance and maybe the final costs rather than a simple refactoring. Remember the discussion about table normalization and de-normalization? It is pretty much the same here. These people are the same that advocate it is better to achieve the application's ROI before any major refactoring. And honestly, 5 years is not enough time for many application to break even the ROI.

Here's a scenario for discussion: A programmer decides to use EF and he maps a common class against the database. He notices that it is very simple IF ONLY he follows 'the yellow-brick-road'. For what I have seen he must follow the EF rules; he must inherit and implement mandatory interfaces dictated by the EF so everything falls right in place. If not, the EF will be just one big expensive fancy feature. Yeah, there is a term for this: Persistence Ignorant. In a glance is like the EF do no adapt to the model, you have to make the model adapt to the EF.

A model should be model ignorant especially nowadays where test-driven development is becoming common in the companies. You can actually test business rules in a higher level. That's why we start to see things like Linq for SQL and binding interfaces. They want to cover the gap left by the persistence ignorance.

The consequence of this is that more and more people are using EF as a data access tool and left to using Linq in the business layer. The business layers will then return datasets within structures called ObjectContext. And good news, the ObjectContext is transactional meaning that you can use System.Transaction to keep the data update rules properties. EF, Linq and ObjectContext: Is this a new implementation being born? Only time will tell but at least they are simple to use, have good performance and gives the programmer good deliverable times...long gone are the days when developers wanted to stay long hours at the office.

Strong points for the EF:

All the query results are objects and you can parse and traverse them in memory without any cost;
There is an embedded conceptual layer where you can do things like denormalize the data structure without affecting much the application.
Linq can be restricted to be used ONLY when needed; Linq is great but it is no silver bullet and people are tempted to overusing it

In a way, these properties are familiar to the typed dataset, aren't they? If you started programming with typed datasets, to migrate to EF is almost natural and has the advantage that now you can isolate a business logic layer with contracts. Translating: gives you scalability. Another good thing is the EF's capacity when compared against other great frameworks like nHibernate.

But it is perfect? IMHO, it is not... and afterall, what is perfection anyway?

Talking about reasons

2008-08-05T04:03:00Z

I was talking to some colleagues today when the subject of 'I could be doing something better, I can do so much more than this…' came up. Of course we all could be doing more and better things, but sometimes we fail to see the good around us.

I recalled something that I've read from the marvelous Argentinean writer Jorge Luís Borges for he has a impeccable tale about this scenario.

One day in India, a superior tiger who is born, soon this tiger is the most feared and respected of all the animals in that area and soon he rules that domain while he is living there. Then it is captured and taken to a common zoo in Europe.

Many moons later, on a peaceful Sunday morning, Dante is at the same zoo and see that tiger. He sees that majestic animal with so much power and respect and it is caged for entertainment just like any regular tiger. So he writes a poem dedicated to that tiger and adds it to the collection called “Divine Comedy”.

Borges says: "All the power, respect and majesty of that tiger, all the knowledge, all the fights he had during his wile life brought it here to this specific morning at this specific zoo to be seen by Dante and to inspire him to write an immortal poem."

I believe, just like that tiger, we all have a reason to be something bigger than us to be here where we are today, this specific morning, at this specific place. For this we just have to continue fighting the good fight and soon our mission will be unveiled.

The economics of software performance

2008-07-22T05:59:00Z

It does not matter the current state in which an application is, everyone always wants a bigger, better and faster version. Yesterday talking to my peers we discussed this point: If you have to choose amongst these items, which one would you leave for last? I said performance.

My point is: To design for high performance is expensive and on these times of ROI is a good thing to save some for when we need it most.

In my first job about a decade ago, I had to work with protocols and microchip programming. In one of those projects I had to implement a little protocol to communicate 2 heaters. I asked to use C, after all I was studying it at the Uni.

At the same time, there was a group of cool people programming in Assembly and these guys told me to give Assembly a try, for performance reasons. I told them:"Guys, I do not have experience with this and I prefer to use C. It will be even better for my grades in the future tests."

Truth must be said; the prototype was done and indeed it was too slow. Way too slow.

It was during my young years and I still remember all the comments I've heard about this bad performance. Those comments just fired me up! I was in a mission now to prove them wrong.

I asked for more time, and with more 2 friends we debugged the code. We were wasting too much time during the handshake. That's it! we found the bottleneck! I said. So we implemented just that handshake part in Assembly.

After some fine tuning: the application was now too fast!

Nowadays, I see people putting too much effort in new technologies, new methodologies and focusing too much about performance where is not really needed. use datasets or use MVC? Use an array list or a generic list? That’s something like over-engineering an application. IMHO, this can lead to complex and expensive systems to maintain.

So, lessons learned from this situation:
First, make sure you have freedom to use your skills in the area you know most. Sometimes we are forced to follow an already designed specification and there is not much freedom for our own ideas. The project priorities supersede our own priorities. If you have this ability then...

Second, do proof of concepts. When you find a proof that is modular enough to detach and that works good enough above the average, move on to the next module...and then...

Focus the effort in time and money with the most critical parts.

Be sure of that: Not even Jesus pleased everyone. At the end it does not matter how effort you placed in the application or how cool you did that module using a new logical path there is always people telling you it could be done better.

That's software, that's simple and that's life.

but hey, these are the sort of comments that makes you want to be better, to improve and move stronger ahead, aren't they?

The 300 Model

2008-05-28T05:36:00Z

"Surrender, we are many, you are few. We won't lay down our weapons, you want them? Come get them! The Persians arrows will blot out the sun. Better this way, we will fight in the shadow."

The movie ‘300’ is like a big punch in the stomach and you have no time to fall. In '300', you see a comic book story that meets the entrepreneurship that meets the world history about the legendary battle of Thermopylae during the year of 480 BC where only 300 Spartans fought against millions of Persians for the future of Greece. Nobody knows exactly how many were really involved in the war; but everyone agreed: It was few against many!

Based on Frank Miller's graphic novel, the movie in its first week generated USD 70 millions in USA alone, with no Tom Cruise, Brad Pitt or a big-shot Hollywood star; instead the major roles were 'unkowns''; a Scottish called Gerard Butler as King Leonidas and a Brazilian called Rodrigo Santoro as King Xerxes.

Some might say that people go to see this movie because it shows sexy appealing scenes, rock music, violence and amazing special effects. That's a lie!

This movie captures the audience because it is about the eternal battle between the strong and the weak, between heroes and villains, between common people and powerful kings. This movie attracts people because every single one wants to know how to defeat a much powerful competitor when we have only too few resources; how to conquer the market when we have few people; how to make more out of few.

‘300’ is a bloody battle between a little small company carried on the shoulders of very talented warriors committed by the honor and a huge corporation machine maintained by millions of regular employees who, some of them, do not even know what are they doing in the company or why they are doing that.

This movie offers much more insights for your professional life than many MBA courses around there. Save yourself USD 500 and watch the movie for a few dollars.

You will learn :

a lesson about the leader who inspired others (even to admire him until his last breath),
you will learn about how to start a company where everyone wants to work on 24x7 (until their last moments),
How to create a brand, a history, a legend (that will overcome the death).
When the captain decides to punish the soldier, Leonidas - the Spartan King - teaches the manager the value of respect towards the lower ranks.
When in doubt of his own convictions, Leonidas asks his partner’s opinion, wife, and Queen - who always is treated equally as any other man - about how to deal with the business.
When busy, just a few days before facing millions and certain death, he finds himself quality time to spend with his little son and teach him the secrets of the business world that he knows so well.
When ready for battle, tie, computer and mobile, everything he does is to KEEP FOCUS ON THE REASON to make the best for the business and his employees.
When King Xerxes, try to seduce Leonidas with the benefits of a possible merge & acquisition of the little Spartan company by the mega-corporation Persia will bring to the shareholders, for him (by becoming CEO of Greece, palaces, women, higher salaries) and for his employees (who will set themselves free from slavery), Leonidas reminds Xerxes that the true slavery of the human being it is not the economical slavery or social, but is to lose the Spartan life style, their culture, their philosophy and the right to take decisions freely without need to justify to nobody.

In every scene you can learn something about you!

Forget the popcorn and the soft drinks, concentrate on the dialogues.

When the Spartan board of directors decides to surrender to Xerxes by not supporting Leonidas, they take the army command out from Leonidas, and tell him to kneel and behold the invisible; Leonidas instead, he does a spin-off in the army, creates himself his own company, he gathers 300 of his best soldiers and go to war against Xerxes' millions.

When he is asked about how he will defeat millions with just 300 soldiers he says he will use his brain when Xerxes uses his ego. He moves towards Thermopylae, a narrow passage where it will neutralize the enemy superiority in numbers.

When an incapable and mediocre Spartan asks him to join the army, Leonidas tells him to go home; in the other hand, Xerxes is sending his mediocre to the front to die first. In Leonidas' small company, only the best, the braves and the brilliants are accepted. He left out the weak, the average, and the non-team players.

Interesting, he brings all the married men with at least one son, meaning these are the most committed people when compared against young and singles. He does not want anybody to slow down his bests and show weakness to the enemy. He can not afford it.

During a very important moment of the battle, Leonidas meets Xerxes, the God-King asks: "How do you envision defeating me? I would kill anyone of my own men just to have you killed"; "I would die for anyone of my own men" Leonidas says.

When Leonidas notices that he will not be able anymore to defeat Xerxes' army, he decides to build a brand!!! He sends just one of your soldiers back home with a mission to make sure everyone knows about of what happened there and the history and glory of those 300 Spartans will live forever. 2.500 years later in the other side of the planet here we are talking about those 300 warriors. "The whole world will know that a handful of free men fought to their death against the tyranny, the whole world will know that few stand against many."

The marketing started by Leonidas worked. This viral inspired millions of Spartans, and produced the necessary proud to unite the Greek world who later ended the Persian invasion and Greece was never defeated by them in this war. And because of that, a new regime was being born, the Democracy.

The Spartan society left nothing behind except the stories of their battles. No art, no music, no dance, no knowledge, no invention, no technology to enlighten the path...they left the unconditional dedication and the strong discipline of his people to become the best warriors ever seen in this world as legacy. Sparta did not produced an Einstein, a Michelangelo, or a Leonardo da Vinci, but they gave birth to the expression "Spartan Life" that is today related to the lifestyle where a person give up privileges and personal benefits to embrace (with body, mind and soul) and surrender his life to a higher cause.

How to Make Productive Project Meetings ?

2008-05-28T05:18:00Z

Project Meetings can be very productives but also can be a real waste of time and money.

Recently while working on a client where I was responsible to have a project development meeting as meeting coordinator. The group of participants were an heterogeneous group and despite the fact that I did not know some of the atendees, the meeting was a big success.

During a conversation on our coffee break I was asked about meetings strategies and how to conduct them.

So I am going to share with you guys here what I told them, and what I effectivly did during that particular meeting:

Every meeting MUST have 3 elements: purpose, agenda and maximum duration. If any of these items is missing, the meeting is meaningless and should not happen.
Make sure you are able to define a purpose for the meeting in a maximum of 2 sentences, for instance:"This meeting is to plan the new developments for the project X". This way, everyone will know why they are there, what needs to be done and how to proceed in order to well-succeed.
Define a clear agenda in advance. Make a list of all the items to be discussed, revised, analysed, displayed etc. When I conduct meetings, my personal strategy is to allocate a time limit for each item in the agenda and to assign the responsability to lead the discussion to someone in the group. Works as a charm.
Define a duration for the meeting, how many minutes/hours it should last. From the start make crystal clear to everyone what time the meeting will start and, sometimes more importantly, when it will end. It is amazing the number of managers who have absolutely no control of their meetings and do not know how to enforce the finishing rule. If you think you have this habit...CHANGE THIS !!!
Do not wait for the delayed people. Meetings must start on the agreeded time. Do not wait about late arrivals. Do not wait for those who need to be called for the meeting. You just make sure everyone gets notified, then when someone arrives after the meeting have started, DO NOT STOP TO REVIEW WHAT WAS SAID. Do this as a proof of respect to those who arrived on time.
If the meeting's organizer is late, Consider the meeting cancelled, and get back to work. How long is considered late? Depends on the company, but I would not wait more than 5 minutes.
Document your meeting. What I do is to put someone in charge of writing down the notes. What to put in the meeting notes? Basically the name of the attendants, the discussed subject, the agreed points, the next developments and/or actions with dates and their respective responsibles.
When the meeting is over - do not wait more than 24 hours - the meeting notes must be sent to: All the participants, to those who could not make it to the meeting and to those who might be influenced by upcoming decisions.
Keep the focus. Every meeting must have a regulator to notify the others when someone is discussing any subject outside the scope of the current topic. Ask one of the presents to volunteer for this task when the meeting is about to start. His/her task is to interrupt the meeting at any given time when the focus is lost and bring back the main subject. This new outside topic can maybe then be noted and even can be discussed in future meetings. In case of doubt regarding a specific topic being in or outside the scope, the meeting organizer has the final word.

I hope these notes can be of any help in your next meetings. If you have any comments or other meetings ideas, please feel free to leave them here and share as well.

See ya later.

Why hiding my wireless Internet SSID will not make my connection safer ?

2008-05-21T04:10:00Z

This weekend I went to a friend's house and we talked a lot about photography, another passion of mine. So I decided to use my iTouch to show him my online portfolio:

- sorry man - he said - I have to tell you my home network name. Otherwise you won't see it available to connect.

- what do you mean? Do you hide your wireless SSID?

- Yeah, I do this for security reasons.

And yet again here we go, another old security myth: Hiding your wireless SSID makes your home network safer.

First of all, what is this ?

In your house you may have Internet access. If you have a laptop what the people normally do is to buy a router, then connect the Internet cable to the router and then the router will 'emit' the signal to the air. This will allow you to connect to your Internet from your bedroom, kitchen, talk to your mother over MSN while walking around your house etc. Everything wirelessly, as long as you can still get the signal. And for that we give a friendly name to this signal called SSID, so you know where to connect.

The thing is, everyone who has a computer with wireless connection also sees your signal.

So how to avoid them to connect to your Internet and make them surf by stealing your connection? Well, you set a password to connect to your router, so when anyone try to connect they will be asked for it.

And here we get to the point: the SSID is not a password. As a matter of fact, the SSID was designed to be public, yes. So by making it public or hidden it really does not change much the security scenario. And besides remember what all the security experts say: there is no security by obscurity. Just because it's hidden it does not mean that it is safe.

So he decided to hide the SSID. Ok. It does not matter, it is not much hidden anyway. Let's see.

The wireless network that you have at home send packages of data-to-air, some are encrypted, some are not, and inside those who are not encrypted they also contain your SSID name. Simple like that and written in plain-text.

So if I am a hacker, I could use a sniffer program to capture the packages and open up to see what's inside. A lot of them I would see crazy stuff, those are encrypted; but in some of them I would see things of the like: trying to connect to SSID name 'myhomenetwork'.

So there we go, our secret is now gone. Do you still think you're safer after that?

Can you reduce the amount of packages without encrypted information? Yes, but you can not stop them 100%, so at some point they will be sent.

Another thing to worry about. If you use Windows XP we can observe an interesting behaviour. If your SSID is hidden, but the laptop is connected to the Internet, XP still apparently keeps sending requests to join the network, continuously. And guess what? The router will reply to your requests using non-encrypted messages.

Funny thing. If we think about it what we are doing here is make our hidden network sends over and over and over again a bunch of replies with not encrypted data with your so-cool-and-hidden SSID.

Why Windows XP and Windows Vista behave like this by default? because SSIDs were, as I mentioned before, designed to be public and I my guess is that Microsoft did this to comply with some governments cyber-laws. I've heard that in some countries, like the USA, it is a crime to keep your SSID hidden and to use hidden identities and hidden networks... all that stuff. I can't confirm that, so it is a guess, but it makes sense to me.

Hiding the SSID won't hide you from the wireless world. Unfortunately people still relates hidden things with secure things.

So how to make my wireless Internet at home more secure? Use something called WPA/WPA2. That's good enough mostly the times. If you are using WEP, change to WPA2. If you are running Windows XP and you have applied all the updates, you are safe. For Windows Vista is even easier because WPA2 comes with it out-of-the-box.

Here some stuff about securing your wireless internet

See you later

What do you do after speaking to a client or customer?

2008-05-21T01:57:00Z

I am an IT guy. That's clear but often I think about many situations from a salesman's point of view, trying to view the world from another angle that I sometimes have no idea how it could be.

What do I mean by that? I'll explain but before please let me share with you this.

I must recognize, even thou IT is a really cool area to work and even thou because the pace it is so demanding we must run everyday just to remain in the same place, unfortunately not always we have the chance to deal with cool and state-of-the-art technology. Yes, sometimes we have to deal with repetitive tasks, sometimes boring tasks, sometimes old products etc. If you are an IT person you know that and might agree with me. Legacy base is a consequence of this fast paced world, as well.

In those situations we wonder: Mate, I am doing this because it was handed over to me out of nowhere and I know this is no rocket-science and despite that I must finish it by tomorrow. I bet we all at some point of our carreers dealt with this scenario, which normally leaves a strange taste in our mouth telling us there is nothing new to be learn from that experience.

Fear not my friends, there is always something to learn does not matter the scenario.

Now, let's get back to the sales person thing I was talking about.

The sales guy after a contact, client visit, sales performed, email delivered to a potential customer, whatever the reason, there is something they must do which is ask himself : What could I have done better ?

Sounds easy and trivial but that's a hard thing to do and as a matter of fact that's something I am trying to do with myself: What have I done today that I've could do better ? What I've done wrong today ?

As the time goes by this becomes a habit just like drinking coffee at 3pm and soon you'll picture yourself in a state of eternal improvement, or at least awareness of it. I am not telling you this is a magic rule to follow in order to achieve the perfection, far from it; but it certainly does something to us which IMHO is a must for a better version of ourselves: It takes us out of our confort zone.

Yet there are people out there who pay for this kind of professional service, Personal Coaching. Honestly, would be great to pay for one of those but I still prefer to put my hard earned money into my mortgage or my kid's school fees. So why not we become our own Personal Coach?

How do I do?
I ask myself: What could I have done better? and I write them on paper. I make a list. I put them on paper because I want that document to be a reminder, and you know what? writing it's free and doesn't hurt, specially the bad things and mistakes we made. Yes, the mistakes are important also because they will be like beacons in this dark ocean of our tries, but I try not to concentrate too much on them after all mistakes are consequences of tries. If you do not do many mistakes it means you haven't tried enough.

Just to illustrate look at our mailboxes with lots of emails trying to sell us stuff. Pay attention to them, I could say that the vast majority is really badly written, from the sales point of view of course. Lots of information about the product requirements, features and prices but very few information about how it would make my life easier or things like why I should buy it now and save effectively 1 hour of coding everyday.

The truth is: very few of them talk about benefits. Very few of them mention how their product will help the customer with its problems.

So, here it goes a good exercise: Try to find out what else that message wants to say in the email selling you stuff. Why I should go for this product instead the competition? How would you write the message to appeal to people like yourself. And how to put yourself ni other situations out of your comfort zone? Try to think about markets you don't know much about, like think how would you manage that coffee shop. If you were an attendant how would you receive a client like yourself looking for a good coffee during the working day?

Excellence is not a point to reach, it is a trajectory made up of very very small baby steps. Hundreds of them taken one at a time, one each day.

See you later.

File Upload and Canonical Issues

2008-05-02T03:57:00Z

Never trust the user input. The incoming data can be the source of many devils and a security flaw can be there just waiting for the right moment and the right person to break your application.

After finishing my upload control I finally did the integration with the website. Now the users can select the files and send it to the website to be processed.

What are the security risks here? Something that can be called 'canonicalization issue'.

For a start all data can be seen on its canonical form. A canonical form is the most simple and most stardard form that any data can be represented, thus canonicalization is the process of converting the data to its canonical form.

Proficient JavaScript programmers are very aware of what I am talking about, and as a matter of fact in our system the user can search for a name using wildcards. So you can ask him: "Retrieve me a list of all the instances where its canonical form includes Bill as mandatory prefix" The user will probably say: "Retrieve what???" but if you ask them: "Give me a list of all the users where their names start with Bill" they will type in the system 'bill*'. The user normally does not know that but he is doing is performing a 'type of canonical query'.

Now, back to our file upload issue. A file name is a very common canonical type. You can call the same file as:

thairecipes.doc
c:\recipes\thairecipes.doc
c:\\recipes\\thairecipes.doc
c:\ recipes\thairecipes.doc
c:%3A%5Crecipes%5Cthairecipes.doc

As you probably figured the last one is the issue. Your Windows operating system will recognize the symbols %5C and %3A.

You see now because we are giving to the user the option to save in our system just about any file name he wants to at the same time we are also opening a door for a sort of canonical attack. Remember : Never trust the user. And by user I am not only talking about a person. In our context an user is any entity who uses a given resource or service, and for that matter an user indeed can be another system or another application.

A hacker would think: "how can I break into this site? Does it allow any easy access to any of its resources?". In our case, yes our website must allow the user to upload files.

What to do now? How to handle a file upload to a web server?

Well, first as a general rule you must not design a website that accept just about any file names created by the user and save it like that. As a matter of fact, any input must be validated and sanitized if possible, not only in client-side but on the server-side as well.

A better design: Do not allow the user to save the file in the web server with the filename that he wants to use. Accept the file, keep the original filename somewhere and let the application rename that file with another name and then save it. I would suggest you to use a GUID string for that matter. That way you are not only closing the doors for a possible canonical attack but also you do not give a chance to a malicious user to try to find out the filenames you might have in your server. For example, If a hacker knows that there is a file called http:\\mywebsite\mydocs\clientid1\file1.doc he will try something like http:\\mywebsite\mydocs\clientid1\file2.doc, and then http:\\mywebsite\mydocs\clientid1\file3.doc and so on. By using an internal name rule creation you minimize his surface.

Another thing to observe: You don't have to fight against and defeat a malicious user, probably there can be hundreds of hackers trying to break your code and you are just one guy against them ( and you don't want to have any sleepless nights during weekends, do you? ) They always find a way to break your code. The best option is to minimize their attack surface. Chances are they are going to move on and concentrate their efforts to break a "weaker website" if your site if strong enough for the first rounds of attack.

These would be some instinctive considerations and additionally I would suggest to take a look at implementing File I/O guidelines as well. At the end of the day, it all depends about how secure you want to be, how much time you have available to implement it and how rigid the specifications were given.

See you later.

The Risk

2008-05-01T04:27:00Z

Last night I had a conversation with some new friends here in the Beautiful Australia and I would like to share something with you guys to reflect upon: "Those people you see, on the National Geographic or Discovery Chanel, who try to reach the top of the Everest, are they crazy or what ?"

Now, picture this. For many decades, many men and women have tried... and there, they lost their lives. As a matter of fact, most of these people were not even able to return to their loved ones. Despite this tragic statistic, more and more people tried... risking... knowing that they probably will die during this journey, leaving family, children, wives behind.

So, why they still did/do this ? Because it was worth it doing it. In the very risk, a transformation happens in you, something is born inside you. No other experience can give you this. It only can be developed in the risk. That's the beauty of the risk.

So, here we are, talking about the idea of "live your live to the most", no regrets, no looking back. Then another question arise: "Can you remember when did you feel most alive?"

For some, it was when they managed to speak a second language for first time, for others was when they first did bungie-jumping in New Zealand or Switzerland, for others was when their first child was born. Now, if you remember "your special moment", I bet a shiver will run through your spine and your heartbeat increase.

If I ask you to describe this feeling, you will be talking about this moment, proud of yourself and about how cool was it. I can almost tell a pattern where people will :

Talk about a scenario in which the risk pushed themselves out of their comfort zone and;
The outcome of the risk is not the main goal, instead the journey is what they remember most and;
They finish the story with a big smile and sparkling eyes.

So, there it goes. Think again about that first question I mentioned. Now, my idea is that the beauty of the risk doesn't lie in the final result - it lies in who you become in the process. Confident. Engaged. Alive. I risk to say more: this is not something you do every summer - it becomes a habit, and it can be contagious. It gives you a new approach to life. After doing it, your life is just not the same anymore, otherwise you will feel stagnant, bored. So, just like that famous question:"when was the last time you did something for the first time?"

See you all later!

Uploading files and raising events

2008-04-19T12:14:00Z

This week while I was awaiting for a new project confirmation I saw myself doing some code fixing for another project, that's something quite common and I really don't mind that. One of my tasks was to develop a little page to upload images to a server. That's not a problem, so I decided to try something new. Yes, I was very keen to use Silverlight for the task, but I used some good old javascript and a very cool concept similar to the called bubbling events, which is basically to raise an event from a determined and let it propagate up in the chain until it is captured.

My first idea was : I will create a web user control that can be easily added to a common web page and this guy will pass the list like a bubbled event.

This control will implement an interface to select and hold a list of files and will be able to send that list to the page and notify the page the list is ready to upload. Let's call it MyUpload control.

being a user control any programmer can then drag and drop this guy to any page and it must be atomic enough to not be too attached to the parent. So it inherits from usercontrol, and will have, at least for now, 2 methods called upload and a click. Here the things are getting cool. This click event is an event raiser.

When a programmer use this control, by calling the click method it will pass the selected file collection to the page. How? sending the list as arguments of the call.

The upload method must then be responsible populate the list and send this list to the caller.

Once the event is triggered in the webpage this page will loop through the items in the received file list and for each instance I will 'save file as...' and provide the path in the web server directory. This will move the file to the desired location as specified in the document request.

Could I save the files in the database? Yes, I could but all the specs for the task are done and that was the mission. So I won't discuss any matters of a better solution for this given scenario.

My upload control will have two buttons add and remove files which will be responsible for maintaining the list of files I want to upload. Why is that? because we want to upload a list of files in one shot and not one by one. Indeed, I'll spend more hours implementing this solution but at the end this will be a reusable control and as I said, any other programmers will then just drag and drop it should they need to implement a similar solution.

How do implement the code for add and remove files? That's the most difficult part and requires a good amount of javascript knowledge. Just this javascript routine can be subject of a post by itself. For now let's say the control has a private method called buildjavascript which renders in the page all the javascript methods necessary to maintain this object list.

Returning to our code behind, this list will be a collection and the best way to keep it and pass it by posting is to make it inherit from a very cool and useful class for this situation: the HttpFileCollection.

The upload button in the control will call the upload method which does nothing but raise the event to the page. The page who has the control implements already the method expected to parse the list as an HttpFileCollection object. Now, we only have to save the object as new name, being in a new location.

My initial design is done and I am doing the javascript side of things. This week was hectic but once things come back to normality and some time is left for me, I will share my code here somewhere.

See you later.

Why adding more memory won't fix your Out of Memory error ?

2008-04-05T15:52:00Z

Here an interesting case. Consider there 2 scenarios:

Both are running the same website, both have the same amount of users connected.

Now imagine this website has a page to upload pictures, just like any regular photo-album website.

For some reason, at some point the users complain that they see an error page indicating out of memory error.

So, you wonder: How come? they are just uploading a photo to my website, and I still have plenty of memory in my server anyway.

Anyhow, you stop thinking about this and go for the easiest, quick and dirty solution: If the system tells me that my computer does not have enough memory then I just need to add more memory. Right?

And guess what? you still will get the error message.

That's a very common mistake. Having a machine with 10GB of memory does not mean you will have 10GB of memory available. I explain.

It does not matter if your computer or server has 512 MB, 1 GB, 2 GB, 4 GB or 8 GB of RAM. If your machine is a 32-bit machine it will only be able to see/manage 4 GB. That's mathematics, that's life, that's the way things are and you can't do nothing about it. A 32-bit machine can not do more than that.

Additional memory may increase your system performance, but it won't increase the memory availability. Sure your computer will use less the hard disk for swapping operations and will be able put more stuff in memory and start some programs faster, but 4GB is the limit; after this point the memory management module will start doing disk swap and to use the famous page file.

And here comes more bad news: Your Windows system on a 32-bit machine requires 2 GB allocated only for it.

So, if you have 4 GB installed, effectively you will have 2 GB only for applications; your windows will be using alone 2 GB.

So, what does out of memory means?

Well, according to some people at Microsoft, this limit for an average configuration is reached between 600 MB and 800 MB of utilization. That 800 number is NOT A RULE, is a baseline. Generally speaking the largest majority of configurations with website, .NET and SQL Server database might have a problem around this point. Of course, this can vary from system to system...as a matter of fact a system can be out of memory at just 600 MB.

Yes, it does sounds crazy. You look so happy now that you just bought a 4GB RAM notebook and your computer is breaking with just 800MB, hun?

Here is another point for you. Have you ever seen someone bragging that he/she bought a 10-megapixel camera and now he/she believes their pictures are going to be better because of this?

Well, guess what? Just like the number of megapixels in a camera box does not have much to do with picture quality, RAM memory does not have much to do with hard disk space.

That's a common mistake: People buy RAM as if they were buying a hard disk.

RAM usage needs to me continuous, unlike hard disk. A simple 5MB Microsoft Word document when saved in a hard disk can be split up in hundreds of pieces; When you open this file in memory, the RAM requires those 5MB to be allocated continuously.

Can you see now the reason for the 'out of memory' message?

Yes, it really means 'there is not enough continuous memory to place that file in memory'. Your system might have 2GB of RAM but unfortunately it might be too busy with stuff running and there is no enough continuous memory to put the picture you are uploading.

Yeah, you can not do much but you can buy a 64-bit machine then when you add more memory you can really use it more efficiently. And yes, we have Microsoft Windows systems for 64-bit machines.

If you do not want to buy a new system of upgrade you current server to a better version then you should think other solutions in the business process, such as to avoid users upload pictures with more than 1 MB in size to be uploaded.

See you later.

Tips on how to improve cohesion in a software design

2008-03-31T13:23:00Z

For the novice in object-oriented programming one of the biggest challenges faced is the new mindset introduced by the OO concept: to be able to think and to model the world in a set of objects and its respective methods.

Every time I take part on a project where the people working are not used to this programming style (using the Microsoft Solution Framework of course) I notice that the main question marks are: what to aggregate? what to specialize? What to move to a super-class? what to leave as module specific operation?

Why do the people have this problem?

IMHO and as far as my experience tells me, I would say 80% of these issues are related to the fact that they are no longer be using the terrible global variables in their programming. and we know how hard is to let go this bad habit once you are used to it and specially if they are coming from a language where the untyped variables can be freely used, as in the classic Visual Basic.

And because those guys now are not using global variables anymore the first effect we notice in the development metric the low cohesion. The cohesion is what is going to bind in the relationship between object and its own operations, making it (the object) responsible for their actions and at the same time not having to worry about who is going to be calling them (the methods). A high cohesion is generally a very nice thing to have, because it introduces a resposability layer that only the methods will be aware of it. For instance: the process of getting a value (i.e.: an account balance) can introduce a change in other places (i.e.: some internal counter limiting the number of requests per minute) and the caller should not bear the responsability of this change.

For these people and these cases I usually show a little trick that now I share with you guys.

1st: Think of the data and its operations like eternal lovers. They are always together in the same domain, they always walk around as a pair no matter where they go. As an exercise take a moment and put everything you are about to develop under a helicopter view, try to draw a diagram grouping the objects and its methods like the one below:

2nd - if you are using too many nested 'if', you probably should be using 'cases/switches'. If you are using too many 'cases/switches' you probably should create an enumerator for those values. If a value needs to be accessed from somewhere outside the scope, create a method just to return this value. This technique outlined here in the 2nd rule is proven to be a great public variable killer, thus helping to enforce a good cohesion.

Now, let's go to the cohesion point. Enumerate the objects you are going to deal. Enumerate the methods you are going to call. Draw a matrix with them in a way that you can see who (object) is calling who (methods), like the one below:

Cool, hun? Now let's analize the data and for that you just have to remember your high school days: That's a popularity contest.

Pick the most popular methods. They will give you a good direction of the objects that are candidates for high cohesion, and even maybe to inherit from a future super-class opening a space for a better model abstraction.

I know this is not a 100% rule but everytime I do this exercise with my colleagues we always find the light at the end of the tunnel. If you have other suggestions or if you do this analisys in a different or not usual way, please share with us.

See you later.

SAFECode and Security Best Practices

2008-03-11T03:53:00Z

hi people,

Do you remember the last time you have developed an application with absolutelly no worries about security ? when you could trust the user input? when you were not worried about sql injection attacks? string attacks? dropdown list modifications? You are right, (if you ever did this) probably it must have been a long long time ago.

Security nowadays IS PART of the regular software development application. It is no more something to worry about only when the application is deployed or attacked. Security is a development-phase concern. Security skills are part of the normal software developer skills and if you are a developer and it is not yours, start learning about security right now.

The thing is: How and where to learn about security? What are the best practices for security in software development? What if I am a Java programmer, Microsoft recommendations apply to me? I use SAP, why should I worry about security?

My friends, I present to you (or at least to those who haven't heard of it yet) SAFECode.

SAFECode or Software Assurance Forum for Excellence in Code is a consortium formed by many big names in the world, including those I told you before. People like Symantec, SAP, EMC and of course Microsoft are part of it.

Together those companies created internal teams and they interact with eachother using SAFECode as an organisational interface and they as a whole share and develop best practices to be recommended to write safer and better products.

Last week SAFECode released their first whitepaper with the best practices for a better and more secure software. That's a very interesting document. Why? because if you take your time to read it, you will notice how similar are the methodologies of those companies.

Hopefully this initiative will grow better and stronger. Maybe one day we will not talk about Microsoft Recommended Best Practices but we will start referring to an Universal Recomended Best Practice? Only time will tell. (and the market forces, of course)

If you are interested take a look at the document here and let us know your ideas ok?

See you later.