Fairness in Assessment

As I was doing some research on fairness in assessment, I came across a very interesting article. An article that I clearly relate to. The name if the article is "Fairness in Assessment" http://www.cs.nott.ac.uk/~smx/PGCHE/fairness.html which highlight the issue of whether it is fair to grade students at the same level or should the cultural background be considered as a factor that might affect the performance of the students. This article talks specifically about students whose first language is not english. Me, being a non-English native speaker, I can say that learning English was not easy. I had always liked the language, but when it came to actually learning it, I had a very hard time, especially when it came to the writing part, because when I translated sentences, it made no sense. Also, for me it was double the work as a english native speaker, because my thoughts came in spanish and in my mind I had to translate it and the even harder part was to try and make sense when I spoke, or wrote it. For me, it was not only having to become accostumed to a new language, but also to a new "world" to me.
As I read the article, many questions came to mind, questions that are so hard to answer by reading articles and readung the book, because teachers have to be fair when it comes to grading his/her students, but what happens when a students first language is not english? is the teacher supposed to be a bit of a hard grader on those that english is their first language? Another fact that is mentioned in this article is that students compare their work, and what is supposed to happen when two ro three students think that they did a very similar work, yet received different grades? So as teachers, what are we supposed to do or better yet how are we supposed to explain that we have graded them based on their english level? and my final question still is, how are we as teachers try and "seem" fully fair?