Plagiarism today III

Things like this drive me nuts.

Even if you take someone else’s work, how hard is it to include a link? A name? Any sort of mention of the original author(s)? I don’t have a beef with this when someone does it with my stuff because most of what I post here is in the Public Domain anyway. But most others do expect some sort of credit.

I frequent a website called Code Project, which programmers and those in the tech industry are quite familiar with. It’s a pretty big site with many, many articles and several orders of magnitude more visitors. Countless members comment, critique and contribute (and occasionally spam) the message boards and article comments all the time.

So then one has to wonder, why would someone risk public ridicule and disparagement by blatantly copying someone else’s work and posting it on this very popular site. I would have missed it too if I hadn’t gone back to an older newsletter and seen this article listed.

Enter Mr. Ram Balak Sharma

On his Code Project profile, he lists his occupation as a .Net Architect. What does that mean anyway? Does it mean programmer? Developer? Manager? Self-titled guru? But I digress…

LOL "passion"!

Mr. Sharma’s credentials include being a “Microsoft Certified Application Developer”. Apparently this is a byword term for not being above blatantly copying posts from other sites and presenting them as your own. His article “Security in ASP.NET MVC by using Anti-Forgery Token” is a complete ripoff of “Anti-Forgery Request Recipes For ASP.NET MVC And AJAX” by Dixin a year earlier. Note : No where on Sharma’s article does he list any other source.

Also note, unlike Mr. Sharma, Dixin doesn’t call himself an “Architect”. He’s just “a UI designer and knows a little bit about programming”.

Proving once again that the magnitude of someone’s espoused prowess is usually inversely proportional to true skill.

TL;DR

Here are both articles compared side by side.

Copy pasta!

I was browsing his site and thought if he’s brazen enough to do this on Code Project, surely he’s done it on his own site. Sure enough, “How to convert List to Datatable in C#” is a rip off of AroLibraries IEnumerableExt class, which is MIT licensed, but good luck finding a single credit on his site. And I don’t see his name listed anywhere on the project “People” section.

If someone from Code Project comes across this post, please send this joker a message and ban his ass.

In case Mr. Sharma or other plagiarists come to wonder how they always, always, ALWAYS end up being outed, I would like to leave you with this helpful message…

...And I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy My brothers.

Update

Some more nuggets…

Original : “jQuery DataTables and ASP.NET MVC integration”
Copy: “JQuery Table in Asp.net MVC on dropdown selection changed”
Excerpts taken without credit, but at least he used different images and modify the wording.

Original : “Improving SQL Server Performance”
Copy : “How optimized sql stored procdure Performance?”
Dude, MSDN?! Really?

Advertisement

Autocomplete with jQuery and MVC

This is just a prelude to a complete spellcheck addon to the discussion forum. I figured I’d start with basic autocomplete first that ties into the wordlist.

All spellcheckers essentially refer to a global wordlist in the specified language and any words that don’t belong, get flagged.

The hardest part of this turned out to be finding a decent wordlist. I was actually surprised at the delicate balance between finding a “good enough” list and one that’s “too good”. Too good? Yes, apparently a list that has too many words will mean you will get a lot of misses where an apparent misspelling turned out to be an obscure word… and you didn’t mean to use obscure words.

The final list I settled on has a word count of 125,346 and was from the Ispell project which also has common acronyms. Note: This is not the same as Iespell (written ieSpell), although if you Google, “Ispell”, you’ll get “ieSpell as the first result. Ispell lists are available for download at the Kevin’s Wordlist page. I have also combined the 4 main english lists into one file (MS Word). WordPress, strangely, won’t allow plain text files to be uploaded, but allows richtext documents. Email me if you want the plaintext version.

I started with a simple DB table to store all the entries. Since I may also be adding more languages, I also have a WordLang field which can be something small like “en”, “de”, “fr” etc…

Wordentries table

 

I then created an MVC app and loaded each of the wordlist files into the db using a simple function (this can take a while depending on filesize):

public List GetWords(string p) {
	var query = from line in File.ReadAllLines(p)
			select new Wordentry
			{
				WordText = NormalizeString(line),
				WordLowercase = NormalizeString(line).ToLower(),
				WordLang = "en"
			};
	return query.ToList();
}

 

After feeding it a HostingEnvironment.MapPath to the filename, I can use this to load all entries into the list and call a db.Wordentries.InsertAllOnSubmition the result. NormalizeString is another helper function which I will list below.

I’m using a Spellword model instead of directly using the Wordentry object since I may want to extend the returned result in the future and changing the columns in the DB wouldn’t be practical.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace Spellcheck.Models
{
	public class Spellword
	{
		public int Id { get; set; }
		public string Spelling { get; set; }
		public string Lowercase { get; set; }
		public string Lang { get; set; }
	}
}

 

And we’re using a SpellRepository class so we’ll keep the controllers free of too much data access stuff.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.IO;
using System.Text;
using System.Globalization;

namespace Spellcheck.Models
{
	public class SpellRepository
	{
		// DataContext global
		private readonly CMDataContext db;

		public SpellRepository(CMDataContext _db)
		{
			db = _db;
		}

		/// <summary>
		/// Counts the total number of word entries
		/// </summary>
		/// <returns>Wordcount int</returns>
		public int GetCount()
		{
			return (from w in db.Wordentries
			 select w.WordText).Count();
		}

		/// <summary>
		/// Searches a given word or word fragment
		/// </summary>
		/// <param name="word">Search word/fragment</param>
		/// <param name="word">Number of returned results</param>
		/// <param name="word">Language to search. Defaults to 10</param>
		/// <param name="word">Search lowercase field only</param>
		/// <returns>List of spellwords</returns>
		public List<Spellword> GetWords(string word, int limit = 10,
			string lang = "en", bool lower = true)
		{
			word = (lower) ?
				NormalizeString(word.ToLower()) :
				NormalizeString(word);

			var query = from w in db.Wordentries
						select w;

			// Get only unique entries in case we have
			// duplicates in the db (Edited from an earlier "GroupBy")
			query = query.Distinct().OrderBy(w => w.WordLowercase);

			// If a language code was specified
			if (!string.IsNullOrEmpty(lang))
				query = query.Where(w=>w.WordLang == lang);

			// Lowercase?
			query = (lower) ?
				query.Where(w => w.WordLowercase.StartsWith(word)) :
				query.Where(w => w.WordText.StartsWith(word));

			// Order alphabetically
			query = query.OrderBy(w => w.WordLowercase);

			return (from w in query
					select new Spellword
					{
						Id = w.WordId,
						Spelling = w.WordText,
						Lowercase = w.WordLowercase,
						Lang = w.WordLang
					}).Take(limit).ToList();
		}
		/// <summary> 
		/// Inserts a new list of words into the spellcheck library
		/// </summary>
		public void SaveWords(List Words)
		{
			var query = Words.GroupBy(w => w.Spelling)
				.Select(w => w.First())
				.OrderBy(w => w.Spelling).ToList();

			List Entries = (from w in query
									   orderby w.Spelling ascending
									   select new Wordentry
									   {
										   WordText = w.Spelling,
										   WordLowercase = w.Lowercase,
										   WordLang = w.Lang
									   }).ToList();

			db.Wordentries.InsertAllOnSubmit(Entries);
			db.SubmitChanges();
		}

		/// <summary> 
		/// Helper function normalizes a given word to the Unicode equivalent
		/// </summary>
		/// <param name="txt">Raw word</param>
		/// <returns>Normalized word</returns>
		private static string NormalizeString(string txt)
		{
			if (!String.IsNullOrEmpty(txt))
				txt = txt.Normalize(NormalizationForm.FormD);

			StringBuilder sb = new StringBuilder();

			sb.Append(
				txt.Normalize(NormalizationForm.FormD).Where(
					c => CharUnicodeInfo.GetUnicodeCategory(c)
					!= UnicodeCategory.NonSpacingMark).ToArray()
				);

			return sb.ToString().Normalize(NormalizationForm.FormD);
		}
	}
}

To use this, we’ll just add a JsonResult action to our controller. I just created a Suggestions action in the default Home controller since this is just an example.

public JsonResult Suggestions(string word, int limit = 10, string lang="en")
{
	List Words = new List();
	if (!string.IsNullOrEmpty(word))
	{
		using (CMDataContext db = new CMDataContext())
		{
			SpellRepository repository = new SpellRepository(db);
			// 10 results is usually enough
			Words = repository.GetWords(word, limit, lang);
		}
	}
	// Need to use AllowGet or else, we'll need use POST
	return Json(Words, JsonRequestBehavior.AllowGet);
}

 

… And that pretty much covers the backend for now.

To test out to see if the word suggestion works, we’ll do one autocomplete textbox. Just add the jQuery and jQuery UI script files and include the jQuery UI CSS to your layout first and add this to the default view :

<script type="text/javascript">
	$(function () {
		var searchtext = $("#search");
		searchtext.autocomplete({
			source: function (request, response) {
				$.ajax({
					url: "/Home/Suggestions", // Or your controller
					dataType: "json",
					data: { word: request.term },
					success: function (data) {
						// Returned data follows the Spellword model
						response($.map(data, function (item) {
							return {
								id: item.Id,
								label: item.Spelling,
								value: item.Lowercase
							}
						}))
					}
				});
			},
			minlength: 3
		});
	});
</script>
<form action="/" method="post">
<input id="search" type="text" name="search" />
</form>

 

Fun fact : Total misspellings as I was writing this (excluding Ispell/ieSpell names and code) before running spellcheck = 12.

Yeah, I really can’t spell.

Discussion Forum update (Utilities and PostRepository)

This is just a followup with two classes from the discussion forum. I haven’t tested the PostRepository class well yet, but I’ll update it with fixes later. Util is the general utilities class that I’ve used in previous projects. It’s basically for rudimentary formatting, input validation etc…

6:40 AM… Time for bed!

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.IO;
using System.Text;
using System.Text.RegularExpressions;
using System.Security.Cryptography;
using System.Globalization;
using System.Web;

namespace Road.Helpers
{
	public class Util
	{
		/// <summary>
		/// Gets the checksum of a local file or some text.
		/// </summary>
		/// <param name="source">Path to a file or a string</param>
		/// <param name="mode">Checksum mode in sha1, sha256, sha512 or md5 (default)</param>
		/// <param name="isFile">True if file mode or false for text mode (defaults to false)</param>
		/// <returns>Completed checksum</returns>
		public static string GetChecksum(string source, string mode = "md5", bool isFile = false)
		{
			byte[] bytes = { };
			Stream fs;

			if (isFile)
				fs = new BufferedStream(File.OpenRead(source), 120000);
			else
				fs = new MemoryStream(Encoding.UTF8.GetBytes(source));

			switch (mode.ToLower())
			{
				case "sha1":
					using (SHA1CryptoServiceProvider sha1 =
						new SHA1CryptoServiceProvider())
						bytes = sha1.ComputeHash(fs);
					break;

				case "sha256":
					using (SHA256CryptoServiceProvider sha256 =
						new SHA256CryptoServiceProvider())
						bytes = sha256.ComputeHash(fs);
					break;

				case "sha512":
					using (SHA512CryptoServiceProvider sha512 =
						new SHA512CryptoServiceProvider())
						bytes = sha512.ComputeHash(fs);
					break;

				case "md5":
				default:
					using (MD5CryptoServiceProvider md5 =
						new MD5CryptoServiceProvider())
						bytes = md5.ComputeHash(fs);
					break;
			}

			// Cleanup
			fs.Close();
			fs = null;

			return BitConverter
					.ToString(bytes)
					.Replace("-", "")
					.ToLower();
		}

		/// <summary>
		/// Returns the page slug or converts a page title into a slug
		/// </summary>
		public static string GetSlug(string val, string d, int length = 45, bool lower = false)
		{
			val = Util.DefaultFlatString(val, d, length);

			// Duplicate spaces
			val = Regex.Replace(val, @"[\s-]+", " ").Trim();
			val = Util.NormalizeString(val); // Remove special chars
			val = Regex.Replace(val, @"\s", "-"); // Spaces to dashes


			// If we still couldn't get a proper string, generate one from default
			val = (String.IsNullOrEmpty(val) || val.Length < 3) ? d :
			val.Substring(0, val.Length <= length ? val.Length : length).Trim();

			
			return (lower) ? val.ToLower() : val;
		}

		private static string NormalizeString(string txt)
		{
			if (!String.IsNullOrEmpty(txt))
				txt = txt.Normalize(NormalizationForm.FormD);

			StringBuilder sb = new StringBuilder();

			sb.Append(
				txt.Normalize(NormalizationForm.FormD).Where(
					c => CharUnicodeInfo.GetUnicodeCategory(c)
					!= UnicodeCategory.NonSpacingMark).ToArray()
				);

			return sb.ToString().Normalize(NormalizationForm.FormD);
		}

		/// <summary>
		/// Gets an array of cleaned tags
		/// </summary>
		/// <param name="txt">A comma delimited string of tags</param>
		/// <returns>Array of cleaned tags</returns>
		public static string[] GetTags(string txt, bool lower = false)
		{
			string[] tags = txt.Split(',');
			ArrayList clean = new ArrayList();

			for (int i = 0; i < tags.Length; i++)
			{
				tags[i] = DefaultFlatString(tags[i], " ").Trim();

				if (!string.IsNullOrEmpty(tags[i]))
					tags[i] = NormalizeString((lower)? 
						tags[i].ToLower() : tags[i]);

				// Don't want to repeat
				if (!clean.Contains(tags[i])) 
						clean.Add(tags[i]);
			}

			return (string[])clean.ToArray(typeof(string));
		}

		/// <summary>
		/// Gets an array of cleaned keywords
		/// </summary>
		/// <param name="txt">A comma delimited string of keywords</param>
		/// <param name="limit">Limit s the number of tags returned</param>
		/// <param name="tolower">Optional parameter to convert the text to lowercase</param>
		/// <returns>Array of cleaned keywords</returns>
		public static List<string> GetKeywords(string txt, int limit, bool tolower = true)
		{
			string[] tags = txt.Split(',');
			List<string> clean = new List<string>();

			for (int i = 0; i < tags.Length; i++)
			{
				tags[i] = Util.DefaultFlatString(tags[i], "");

				if (!String.IsNullOrEmpty(tags[i]))
				{
					if (tolower)
						clean.Add(tags[i].ToLower());
					else
						clean.Add(tags[i]);
				}
			}

			return clean;
		}

		/// <summary>
		/// Shorten a give text block followed by an ellipse
		/// </summary>
		public static string TrimText(string strInput, int intNum)
		{
			strInput = strInput.Replace("\r", string.Empty)
				.Replace("\n", string.Empty);
			if ((strInput.Length > intNum) && (intNum > 0))
			{
				strInput = strInput.Substring(0, intNum) + "...";
			}
			return strInput;
		}

		/// <summary>
		/// Checks whether string has value or sets default it doesn't or is at 0
		/// </summary>
		public static int DefaultInt(string val, int d, int? min)
		{
			int tmp = 0;

			if (!Int32.TryParse(val, out tmp))
				tmp = d;

			if (min.HasValue)
				if (tmp <= min.Value) tmp = d;

			return tmp;
		}

		/// <summary>
		/// Checks whether nullable int has value or sets default it doesn't or is at 0
		/// </summary>
		public static int DefaultInt(int? val, int d, int? min)
		{
			val = val ?? d;
			if (min.HasValue)
				if (val.Value <= min.Value) val = d;

			return val.Value;
		}

		/// <summary>
		/// Checks whether nullable bool has value or sets default it doesn't
		/// </summary>
		public static bool DefaultBool(bool? val, bool d)
		{
			val = val ?? d;
			return val.Value;
		}

		/// <summary>
		/// Checks whether nullable bool has value or sets default it doesn't
		/// </summary>
		public static bool DefaultBool(string val, bool d)
		{
			bool tmp = d;

			if (Boolean.TryParse(val, out tmp))
				return tmp;

			return d;
		}

		/// <summary>
		/// Returns a flat (no line breaks) string or a default value if empty
		/// </summary>
		public static string DefaultFlatString(string val, string d, int l = 255)
		{
			return Util.DefaultString(val, d, l).Replace(Environment.NewLine, "");
		}

		/// <summary>
		/// Checks whether nullable string has value or sets default it doesn't or is at empty
		/// </summary>
		public static string DefaultString(string val, string d, int l = 255)
		{
			if (string.IsNullOrEmpty(val)) val = d;

			val.Replace("\r", Environment.NewLine)
				.Replace("\n", Environment.NewLine);

			if ((val.Length > 0) && (val.Length > l))
				val = val.Substring(0, l-1);

			return val;
		}

		/// <summary>
		/// Converts a string value to a DateTime object or returns the default value on failure
		/// </summary>
		public static DateTime DefaultDate(string val, DateTime d)
		{
			DateTime dt;
			if (DateTime.TryParse(val, out dt))
				return dt;

			return d;
		}

		/// <summary>
		/// Converts a nullable date value to a DateTime object or returns the default value on failure
		/// </summary>
		public static DateTime DefaultDate(DateTime? val, DateTime d)
		{
			DateTime dt;
			dt = (val.HasValue) ? val.Value : d;

			return d;
		}

		/// <summary>
		/// Gets the current user's IP address
		/// </summary>
		public static string GetUserIP()
		{
			// Connecting through a proxy?
			string ip = HttpContext.Current.Request.ServerVariables["HTTP_X_FORWARDED_FOR"];

			// None found
			if (string.IsNullOrEmpty(ip) || ip.ToLower() == "unknown")
				ip = HttpContext.Current.Request.ServerVariables["REMOTE_ADDR"];

			return ip;
		}

		public static string GetEmail(string v)
		{
			string email = @"^[\w!#$%&'*+\-/=?\^_`{|}~]+(\.[\w!#$%&'*+\-/=?\^_`{|}~]+)*" +
							@"@((([\-\w]+\.)+[a-zA-Z]{2,4})|(([0-9]{1,3}\.){3}[0-9]{1,3}))$";

			if (!string.IsNullOrEmpty(v))
				if (Regex.IsMatch(v, email))
					return v;

			// Didn't match the email format, so sent a cleaned string
			return Util.DefaultFlatString(v, "");
		}
	}
}

PostRepository. This one’s a bit long…

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using Road.Helpers;

namespace Road.Models
{
	public class PostRepository
	{
		// Common DataContext
		private readonly CMDataContext db;

		/// <summary>
		/// Constructor
		/// </summary>
		/// <param name="context">Global context</param>
		public PostRepository(CMDataContext context)
		{
			this.db = context;
		}

		#region Topic display methods


		/// <summary>
		/// Gets a topic by the given id and finds corresponding replies if any
		/// </summary>
		/// <param name="id">Topic id to search</param>
		/// <param name="index">Current page index</param>
		/// <param name="limit">Page size limit</param>
		/// <param name="unapproved">Include unapproved replies (optional)</param>
		/// <param name="status">Array of reply status types (optional)</param>
		/// <param name="newestFirst">Sort by newest replies first (optional)</param>
		public Topic TopicById(int id, int index, int limit,
			bool unapproved = false, ReplyStatus[] status = null, bool newestFirst = false)
		{
			var query = from p in db.Posts
						where p.PostId == id
						select p;

			Topic topic = TopicQuery(query).FirstOrDefault();

			// We have a topic and replies were also requested
			if (topic != null && limit > 0)
			{
				var rquery = from p in db.Posts
							 join pa in db.PostRelations on p.PostId equals pa.ParentId
							 where pa.ParentId == topic.Id
							 select p;

				topic.Replies =
					ReplyQuery(rquery, unapproved, status, newestFirst).ToPagedList(index, limit);
			}

			return topic;
		}

		/// <summary>
		/// Gets a list of topics (most basic request, usually for frontpage)
		/// </summary>
		/// <param name="limit">Page size limit</param>
		/// <param name="unapproved">Include unapproved topics (optional)</param>
		/// <param name="status">Array of status topic status types (optional)</param>
		/// <param name="newestFirst">Sort by newest topics first (optional)</param>
		/// <returns>List of topics</returns>
		public List<Topic> TopicList(int limit, bool unapproved = false,
			TopicStatus[] status = null, bool newestFirst = true)
		{
			var query = from p in db.Posts
						select p;

			return TopicQuery(query, unapproved, status, newestFirst).Take(limit).ToList();
		}

		/// <summary>
		/// Gets a paged list of topics
		/// </summary>
		/// <param name="index">Current page index</param>
		/// <param name="limit">Page size limit</param>
		/// <param name="unapproved">Include unapproved topics (optional)</param>
		/// <param name="status">Array of status topic status types (optional)</param>
		/// <param name="newestFirst">Sort by newest topics first (optional)</param>
		/// <returns>Paged list of topics</returns>
		public PagedList<Topic> TopicPageList(int index, int limit,
			bool unapproved = false, TopicStatus[] status = null, 
			bool newestFirst = true)
		{
			var query = from p in db.Posts
						select p;

			return TopicQuery(query, unapproved, status, newestFirst).ToPagedList(index, limit);
		}

		/// <summary>
		/// Gets a paged list of topics belonging to a tag(s)
		/// (This uses the lowercase TagName property)
		/// </summary>
		/// <param name="tag">Array of tags to search</param>
		/// <param name="index">Current page index</param>
		/// <param name="limit">Page size limit</param>
		/// <param name="unapproved">Include unapproved topics (optional)</param>
		/// <param name="status">Array of status topic status types (optional)</param>
		/// <param name="newestFirst">Sort by newest topics first (optional)</param>
		/// <returns>Paged list of topics</returns>
		public PagedList<Topic> TopicsByTag(string[] tag, int index, int limit,
			bool unapproved = false, TopicStatus[] status = null, bool newestFirst = true)
		{
			var query = from t in db.PostTags
						join pt in db.PostTagRelations on t.TagId equals pt.TagId
						join p in db.Posts on pt.PostId equals p.PostId
						where tag.Contains(t.TagName)
						select p;

			return TopicQuery(query, unapproved, status, newestFirst).ToPagedList(index, limit);
		}

		/// <summary>
		/// Gets an individual topic belonging to a list of tag(s)
		/// </summary>
		/// <param name="tagId">Array of tag Ids to search</param>
		/// <param name="index">Current page index</param>
		/// <param name="limit">Page size limit</param>
		/// <param name="unapproved">Include unapproved topics (optional)</param>
		/// <param name="status">Array of status topic status types (optional)</param>
		/// <param name="newestFirst">Sort by newest topics first (optional)</param>
		/// <returns>Paged list of topics</returns>
		public PagedList<Topic> TopicsByTagId(int[] tagId, int index, int limit,
			bool unapproved = false, TopicStatus[] status = null, bool newestFirst = true)
		{
			var query = from t in db.PostTags
						join pt in db.PostTagRelations on t.TagId equals pt.TagId
						join p in db.Posts on pt.PostId equals p.PostId
						where tagId.Contains(t.TagId)
						select p;

			return TopicQuery(query, unapproved, status, newestFirst).ToPagedList(index, limit);
		}

		/// <summary>
		/// Gets a paged list of topics by the search criteria
		/// </summary>
		/// <param name="search">Title and body search terms</param>
		/// <param name="index">Current page index</param>
		/// <param name="limit">Page size limit</param>
		/// <param name="unapproved">Include unapproved topics (optional)</param>
		/// <param name="status">Array of status topic status types (optional)</param>
		/// <param name="newestFirst">Sort by newest topics first (optional)</param>
		/// <returns>Paged list of topics</returns>
		public PagedList<Topic> TopicsBySearch(string search, int index, int limit,
			bool unapproved = false, TopicStatus[] status = null, bool newestFirst = true)
		{
			var query = from p in db.Posts
						where p.BodyText.Contains(search) || p.Title.Contains(search)
						select p;

			return TopicQuery(query, unapproved, status, newestFirst).ToPagedList(index, limit);
		}

		/// <summary>
		/// Gets a paged list of replies by the search criteria 
		/// (only searches the bodytext)
		/// </summary>
		/// <param name="search">Search terms</param>
		/// <param name="index">Current page index</param>
		/// <param name="limit">Page size limit</param>
		/// <param name="unapproved">Include unapproved topics (optional)</param>
		/// <param name="status">Array of topic status types (optional)</param>
		/// <param name="newestFirst">Sort by newest topics first (optional)</param>
		/// <returns>Paged list of topics</returns>
		public PagedList<Reply> RepliesBySearch(string search, int index, int limit,
			bool unapproved = false, ReplyStatus[] status = null, bool newestFirst = true)
		{
			var query = from p in db.Posts
						where p.BodyText.Contains(search)
						select p;

			return ReplyQuery(query, unapproved, status, newestFirst).ToPagedList(index, limit);
		}

		#endregion

		#region Save methods

		/// <summary>
		/// Saves or creates a new reply under the given topic
		/// </summary>
		/// <param name="topic">Topic the reply belongs to</param>
		/// <param name="reply">Reply to save</param>
		public Reply SaveReply(Topic topic, Reply reply)
		{
			Post p = null;
			DateTime dt = DateTime.UtcNow;

			if (reply.Id != 0)
			{
				p = (from post in db.Posts
					where post.PostId == reply.Id
					select post).FirstOrDefault();
			}
			else
			{
				p = new Post();
				p.CreatedDate = dt;
				p.ReplyCount = 0;
				p.ViewCount = 0;
				db.Posts.InsertOnSubmit(p);
			}

			p.Approved = reply.Approved;
			p.Status = (byte)reply.Status;
			p.Threshold = reply.Threshold;
			p.LastModified = dt;
			p.BodyHtml = reply.Body;
			p.BodyText = reply.Summary;

			// Save reply
			db.SubmitChanges();

			// If this is a new reply...
			if (p.PostId > 0 && reply.Id == 0)
			{
				// We now have an Id to set
				reply.Id = p.PostId;

				// Create Author, PostRelation and PostAuthor relationships
				Author a = new Author();
				a.MemberId = reply.CreatedBy.Id;
				a.AuthorIP = reply.CreatedBy.IP;
				a.AuthorName = reply.CreatedBy.Name;
				a.AuthorEmail = reply.CreatedBy.Email;
				a.AuthorWeb = reply.CreatedBy.Web;
				db.Authors.InsertOnSubmit(a);


				PostRelation pr = new PostRelation();
				pr.ParentId = topic.Id;
				pr.PostId = p.PostId;
				db.PostRelations.InsertOnSubmit(pr);
				db.SubmitChanges();

				if (a.AuthorId > 0)
				{
					PostAuthor pa = new PostAuthor();
					pa.AuthorId = reply.CreatedBy.Id;
					pa.PostId = reply.Id;
					db.PostAuthors.InsertOnSubmit(pa);
					db.SubmitChanges();
				}
			}

			return reply;
		}

		/// <summary>
		/// Saves or creates a new topic
		/// </summary>
		/// <param name="topic">Topic to save</param>
		/// <returns>Returns the saved topic</returns>
		public Topic SaveTopic(Topic topic)
		{
			Post p = null;
			DateTime dt = DateTime.UtcNow;

			if (topic.Id != 0)
			{
				p = (from post in db.Posts
					 where post.PostId == topic.Id
					 select post).FirstOrDefault();
			}
			else
			{
				p = new Post();
				p.CreatedDate = dt;
				db.Posts.InsertOnSubmit(p);
			}

			p.Title = topic.Name;
			p.Approved = topic.Approved;
			p.Status = (byte)topic.Status;
			p.Threshold = topic.Threshold;

			p.LastModified = dt;
			p.BodyHtml = topic.Body;
			p.BodyText = topic.Summary;

			p.ViewCount = topic.ViewCount;
			p.ReplyCount = topic.ReplyCount;

			// Save
			db.SubmitChanges();

			// If this is a new topic...
			if (p.PostId > 0 && topic.Id == 0)
			{
				// Set the Id, now that we have one
				topic.Id = p.PostId;

				// Create author and set relationship
				Author a = new Author();
				a.MemberId = topic.CreatedBy.MemberId;
				a.AuthorIP = topic.CreatedBy.IP;
				a.AuthorName = topic.CreatedBy.Name;
				a.AuthorEmail = topic.CreatedBy.Email;
				a.AuthorWeb = topic.CreatedBy.Web;
				db.Authors.InsertOnSubmit(a);

				PostRelation pr = new PostRelation();
				pr.ParentId = p.PostId; // Same since it's a topic
				pr.PostId = p.PostId;
				db.PostRelations.InsertOnSubmit(pr);

				db.SubmitChanges();

				if (a.AuthorId > 0)
				{
					PostAuthor pa = new PostAuthor();
					pa.AuthorId = a.AuthorId;
					pa.PostId = p.PostId;
					db.PostAuthors.InsertOnSubmit(pa);

					db.SubmitChanges();
				}
			}


			topic.Slug = Util.GetSlug(p.Title, "topic");


			ApplyTags(topic.Tags.ToList(), topic);

			return topic;
		}

		#endregion

		#region Tag methods

		/// <summary>
		/// Gets a list of tags by a search string 
		/// (usually for tag autocomplete)
		/// </summary>
		/// <param name="tag">Tag search string</param>
		/// <param name="limit">Page size limit</param>
		/// <returns></returns>
		public List<Tag> TagsByName(string tag, int limit)
		{
			var query = from t in db.PostTags
						orderby t.TagName ascending
						where t.TagName.StartsWith(tag)
						select new Tag
						{
							Id = t.TagId,
							Name = t.TagName,
							Slug = t.Slug,
							DisplayName = t.TagName
						};

			if (limit > 0)
				query = query.Take(limit);

			return query.ToList();
		}

		/// <summary>
		/// Associates a list of tags with the given topic
		/// </summary>
		/// <param name="tags">Tags to link to topic</param>
		/// <param name="topic">Target topic</param>
		private void ApplyTags(List<Tag> tags, Topic topic)
		{
			List<PostTagRelation> existing = (from pt in db.PostTagRelations
											  join t in db.PostTags on pt.TagId equals t.TagId
											  where pt.PostId == topic.Id
											  select pt).ToList();

			// Clean existing relationships
			db.PostTagRelations.DeleteAllOnSubmit(existing);
			db.SubmitChanges();

			// Setup the new relationships
			List<PostTagRelation> newrelation = new List<PostTagRelation>();

			// Store the new tags and get the complete list of tags
			tags = StoreTags(tags, topic.CreatedBy);

			foreach (Tag t in tags)
			{
				PostTagRelation tag = new PostTagRelation();
				tag.TagId = t.Id;
				tag.PostId = topic.Id;
				newrelation.Add(tag);
			}
			// Save the new tag relationships
			db.PostTagRelations.InsertAllOnSubmit(newrelation);
			db.SubmitChanges();
		}


		/// <summary>
		/// Finds existing tags and creates new tags with the associated creator if
		/// the tag doesn't exist
		/// </summary>
		/// <param name="tags">List of tags to create/find</param>
		/// <param name="creator">Tag creator</param>
		/// <returns>List of found and newly created tags</returns>
		private List<Tag> StoreTags(List<Tag> tags, Creator creator)
		{
			// Complete list of all tags
			List<Tag> complete = new List<Tag>();

			// Created date
			DateTime dt = DateTime.UtcNow;

			string[] search = tags.Select(tg => tg.Name).ToArray();
			string[] existing = (from t in db.PostTags
								where search.Contains(t.TagName)
								select t.TagName).ToArray();

			// Tags except those already in the database
			string[] newtags = search.Except(existing).ToArray();

			// We have new tags to save
			if (newtags.Length > 0)
			{
				List<PostTag> savetags = (from tg in tags
								   where newtags.Contains(tg.Name)
								   select new PostTag
								   {
									   DisplayName = tg.DisplayName,
									   TagName = tg.DisplayName.ToLower(),
									   Slug = Util.GetSlug(tg.DisplayName, tg.Name),
									   Status = (byte)TagStatus.Open,
									   LastModified = dt,
									   CreatedDate = dt,
									   BodyHtml = "",
									   BodyText = ""
								   }).ToList();

				if (savetags.Count() > 0)
				{
					db.PostTags.InsertAllOnSubmit(savetags);
					db.SubmitChanges();

					// Create author info for each new tag
					Author author = getAuthor(creator);
					List<TagAuthor> authors = (from tg in savetags
											   select new TagAuthor
											   {
												   AuthorId = author.AuthorId,
												   TagId = tg.TagId
											   }).ToList();
					db.TagAuthors.InsertAllOnSubmit(authors);
					db.SubmitChanges();
				}

				// Get all existing and newly inserted tags
				complete = (from tg in db.PostTags
							where search.Contains(tg.TagName)
							select new Tag
							{
								Id = tg.TagId,
								Name = tg.TagName,
								DisplayName = tg.DisplayName,
								Slug = tg.Slug
							}).ToList();
			}

			return complete;
		}

		#endregion


		#region Queries

		/// <summary>
		/// Creates a deferred execution IQueryable to search topics
		/// </summary>
		/// <param name="posts">Initial search query</param>
		/// <returns>Topic IQueryable</returns>
		private IQueryable<Topic> TopicQuery(IQueryable<Post> posts, 
			bool unapproved = false, TopicStatus[] status = null,
			bool newestFirst = true)
		{
			var query = from p in posts
						join au in db.PostAuthors on p.PostId equals au.PostId
						join a in db.Authors on au.AuthorId equals a.AuthorId
						join m in db.Members on au.AuthorId equals m.MemberId into author
						from auth in author.DefaultIfEmpty() // Empty if anonymous post

						let postauthor = getCreator(a, auth)
						let tags = GetTagsForTopic(p.PostId)

						select new { p, postauthor, tags };

			// Include unapproved topics?
			query = (unapproved) ?
				query.Where(r => r.p.Approved == false) :
				query.Where(r => r.p.Approved == true);

			// Any status other than "Open"?
			query = (status != null) ?
				query = query.Where(r => status.Contains((TopicStatus)r.p.Status)) :
				query = query.Where(r => r.p.Status == (byte)TopicStatus.Open);

			// Sort by new topics first?
			query = (newestFirst) ?
				query.OrderByDescending(r => r.p.CreatedDate) :
				query.OrderBy(r => r.p.CreatedDate);

			return from r in query
				   select new Topic
				   {
					   Id = r.p.PostId,
					   Name = r.p.Title,
					   CreatedBy = r.postauthor,
					   CreatedDate = r.p.CreatedDate,
					   LastModified = r.p.LastModified,
					   Summary = r.p.BodyText,
					   Slug = Util.GetSlug(r.p.Title, "topic", 50, true),
					   Tags = new LazyList<Tag>(r.tags),
					   ViewCount = r.p.ViewCount,
					   ReplyCount = r.p.ReplyCount,
					   Threshold = (float)r.p.Threshold,
					   Status = (TopicStatus)r.p.Status
				   };
		}

		/// <summary>
		/// Creates a deferred execution IQueryable to search replies
		/// </summary>
		/// <param name="posts">Initial posts query</param>
		/// <param name="status">Status restriction array</param>
		/// <param name="newestFirst">Sort by new replies first</param>
		/// <returns>Reply IQueryable</returns>
		private IQueryable<Reply> ReplyQuery(IQueryable<Post> posts,  bool unapproved, ReplyStatus[] status, bool newestFirst)
		{
			var query = from p in posts
						join au in db.PostAuthors on p.PostId equals au.PostId
						join a in db.Authors on au.AuthorId equals a.AuthorId
						join m in db.Members on au.AuthorId equals m.MemberId into author
						from auth in author.DefaultIfEmpty() // Empty if anonymous post

						let postauthor = getCreator(a, auth)
						select new { p, postauthor };

			// Include unapproved replies?
			query = (unapproved) ?
				query.Where(r => r.p.Approved == false) :
				query.Where(r => r.p.Approved == true);

			// Any status other than "Open"?
			query = (status != null) ?
				query = query.Where(r => status.Contains((ReplyStatus)r.p.Status)) :
				query = query.Where(r => r.p.Status == (byte)ReplyStatus.Open);

			// Sort by new replies first?
			query = (newestFirst) ?
				query.OrderByDescending(r => r.p.CreatedDate) :
				query.OrderBy(r => r.p.CreatedDate);

			return from r in query.AsQueryable()
				   select new Reply
				   {
					   Id = r.p.PostId,
					   CreatedBy = r.postauthor,
					   CreatedDate = r.p.CreatedDate,
					   LastModified = r.p.LastModified,
					   Body = r.p.BodyHtml,
					   Threshold = (float)r.p.Threshold
				   };
		}

		/// <summary>
		/// Helper finds the tags for a topic by id
		/// </summary>
		/// <param name="id">Topic id to search</param>
		/// <returns>IQueryable Tag</returns>
		private IQueryable<Tag> GetTagsForTopic(int id)
		{
			return from t in db.PostTags
				   join pt in db.PostTagRelations on t.TagId equals pt.TagId
				   where pt.PostId == id
				   select new Tag
				   {
					   Id = t.TagId,
					   Name = t.TagName,
					   Slug = t.Slug,
					   DisplayName = t.DisplayName,
					   CreatedDate = t.CreatedDate,
					   LastModified = t.LastModified
				   };
		}

		#endregion

		#region Author/Creator Helpers

		/// <summary>
		/// Helper function generates a save friendly Author from a given Creator
		/// </summary>
		/// <param name="c">Creator data</param>
		/// <returns>Author object</returns>
		private static Author getAuthor(Creator c)
		{
			if (c == null)
				return null;

			Author author = new Author();
			if (c.Id > 0)
			{
				author.AuthorId = c.Id;
			}
			else
			{
				author.AuthorEmail = c.Email;
				author.AuthorWeb = c.Web;
			}

			author.AuthorIP = c.IP;
			author.AuthorName = c.Name;

			return author;
		}

		/// <summary>
		/// Finds or creates a Creator object from given author information
		/// </summary>
		/// <param name="a">Saved author information</param>
		/// <param name="m">Optional membership information</param>
		/// <returns>Composite Creator object</returns>
		private static Creator getCreator(Author a, Member m)
		{
			Creator au = new Creator();

			au.IP = a.AuthorIP;
			if (m != null)
			{
				au = getCreator(m);
			}
			else
			{
				au.LastModified = DateTime.MinValue;
				au.Id = a.AuthorId;
				au.Name = a.AuthorName;
				au.DisplayName = a.AuthorName;
				au.Email = a.AuthorEmail;
				au.Web = a.AuthorWeb;
			}
			return au;
		}

		/// <summary>
		/// Helper function generates a Creator object from membership info
		/// </summary>
		/// <param name="m">Member object</param>
		/// <returns>Composite Creator object</returns>
		private static Creator getCreator(Member m)
		{
			return new Creator
			{
				Id = m.MemberId,
				Name = m.Username,
				DisplayName = m.DisplayName,
				Email = m.Email,
				Web = m.Web,
				Slug = Util.GetSlug(m.Username, m.MemberId.ToString(), 70),
				CreatedDate = m.CreatedDate,
				LastModified = m.LastActivity,
				Avatar = m.Avatar
			};
		}

		#endregion
	}
}

ASP.Net MVC 3 Script license WTF?!

I had a few breaks between visits so I decided to re-write some of my old work in MVC 3 and Razor. I was going through all the included files in the Scripts folder when I came across Modernizr. I admit that I haven’t really looked into it that much since most of my client work involves CSS2 and XHTML, not CSS3 and HTML5. And the few times I needed HTML5 compatibility for <video> and such it was included as needed.

For the record, Modernizr is offered under a BSD/MIT license.

Imagine my surprise when, in Visual Web Developer Express 2010, I opened up modernizr-1.7.js to find this on top :

/*!
* Note: While Microsoft is not the author of this file, Microsoft is
* offering you a license subject to the terms of the Microsoft Software
* License Terms for Microsoft ASP.NET Model View Controller 3.
* Microsoft reserves all other rights. The notices below are provided
* for informational purposes only and are not the license terms under
* which Microsoft distributed this file.
*
* Modernizr v1.7
* http://www.modernizr.com
*
* Developed by:
* - Faruk Ates  http://farukat.es/
* - Paul Irish  http://paulirish.com/
*
* Copyright (c) 2009-2011
*/

After that, I looked into the rest of the included script files and I found the same in jQuery and jQuery UI as well and they are originally MIT/GPL . Now I’ve either been living under a rock (actually living “out-of-town” for a while) or this is a sudden inclusion not present in any pre-MVC3 projects because I don’t recall seeing anything like this before.

So Microsoft is re-licensing the original code because it is included with an MVC 3 project template? Can they even do that even though the files are included in their project template (what about consent form the original authors)? Is this only because it was included in the template or does it also apply when I use the original code instead of the provided copies as long as I’m still using it with other Microsoft provided script files?

I really don’t like seeing surprises like this because it’s not a technical problem; It’s a legalese problem and I really hate to have to choose code based on licenses. I’m now tempted to simply delete the entire contents of the Scripts folder and download everything from scratch.

Why are People so complicated?

That isn’t a rhetorical question; I really would like to know.

I ran into an old collegue of mine who had just accepted a gig creating your garden variety customer management app (basically a very light CRM and shopping cart). Normally she would have recommended using something off-the-shelf and customizing it rather than wasting time building something from scratch. But the back end was already so simple, and their need to change things in the front end was so frequent, that she needed to write it herself.

She was describing how she will implement the “Person” classes, when I felt a sudden wave of déjà vu as if I was transported back to a previous lifetime as a Soviet Scientist creating the N1 rocket. She was using the Table Profile Provider sample for most of the custom data fields, which would mean that adding a new field or property would have actually created a new column in the data table. The client would then augment the basic properties by editing the Web.config later on.

From the introduction :

The first sample provider (SqlTableProfileProvider) stores each Profile property in a separate database column. Furthermore the provider stores the Profile data without serializing it, which means that the Profile property type needs to be compatible with the target database column.

The second sample provider (SqlStoredProcedureProfileProvider) maps each Profile property to a parameter on a custom stored procedure. Like the table based provider, this provider expects that each Profile property is of a type that is compatible with its corresponding stored procedure parameter. The powerful aspect of the stored procedure based provider is that other than the requirement to implement some stored procedures with a specific set of parameters, you can implement whatever business logic you need in the stored procedures to map the Profile data to your own database schema and database logic.

OK, so she won’t know all the possible profile options the client would need, but there has to be some she just knows would be necessary. I had to ask her why she didn’t just map all possible combinations of profile properties into a couple of extra tables instead.

Now this isn’t the first time I a girl had given me the “wow, he’s a an idiot” look (I’ve lost count of those, really), but it was the first time in a long time, I felt it wasn’t deserved.

Why are People so complicated?

Two very important questions you need to ask when outlining a person in your app….

Which properties would a person have? And which of those would be more than one?

These two interdependent questions will really determine how you define people in your application since the rest is just fluff you can get away with by using the Provider Model for the most part.

• A person may have several different street addresses for example. Maybe one for shipping and one for billing, and perhaps yet another for separate billing purposes or a separate shipping destination.
• A person may have several different email addresses; one primary, one personal, one for work, one for spam etc…
• A person may have several different phone numbers as well. One for daytime, one for evenings, one mobile, one fax etc…

Since as far back as I can remember, from the day I started programming, I’ve adhered to the age old philosophy : I’m a programmer = I’m lazy. But the good kind of lazy. The lazy that doesn’t want to hard-code properties in case they need to be changed. The lazy that doesn’t re-invent the wheel, but only wants to add better rims.

I’ve never believed that it’s prudent nor practical to access or store complex profile properties using the Profile Provider. While this might seem ironic, you also need to consider that it doesn’t have native knowlege of how you’re storing the data. That how is what should determine whether the Profile Provider is appropriate.

A little complexity now will save more of it later on

Consider the following model…

Membership and Profile model

This is an example layout from the Linq to SQL class using the database I created for the OPAR project. I’ll the other tables and matching classes at a later date.

Instead of creating a new column for each profile property, you essentially implement a property bag per member. The MemberProfileField stores all your data with one entry per field. The drawback is that your MemberProfileFields table will get quite large over time, however you’re not altering the schema in any way which means all your content will still be safe. Which brings me to another programming philosophy…

Try to keep your data out of your provider

Tell your data what type it’s storing but nothing else; In this case, I have a field in MemberProfileFields called “FieldType”. This isn’t int, bool, xml etc… It’s actually textbox, multiline, option and so on. You’re creating a web application, so store what input type you’re using to get and store the data. This is very handy for the future as you can add new HTML form types at a later date. The provider logic dictates which field it will tap to store the data.

And back in your profile, you can have a helper class with the following mappings :

text = <input type=”text” id=”{FieldName}” name=”{FieldName}” value=”{ContentText}” />
multiline = <textarea id=”{FieldName}” name=”{FieldName}”>{ContentLongText}</textarea>
option = <input type=”checkbox” id=”{FieldName}” name=”{FieldName}” value=”{ContentBool}” />
… And so on.

Of course, you can do one better by adding a ProfileFieldTypes table to extend this a lot further and substitute FieldType for FieldTypeId with a matching foreign key as well. By branching out the schema to tables, you can actually dynamically create profile properties without ever having to touch the Web.config .