Archives
-
The sideways stack trace
A common pattern in asynchronous programming is the callback pattern. Libraries such as async make use of that pattern to build asynchronous versions of common control flow structures. One unfortunate possible consequence of this is strange stack traces. For instance, here is code that executes five functions serially:var functions = [
function one(done) {done();},
function two(done) {done();},
function three(done) {done();},
function four(done) {done();},
function five(done) {done();}
];
async.series(functions, function final() {
debugger;
}); -
The curse of the never-ending async method call
Whether you use promises, callbacks, or other abstractions (except maybe the async/await pattern) to deal with asynchrony, control flow is going to get harder to follow. Stack traces are typically meaningless, and finding what went wrong can take substantially more time. One thing that very often goes wrong, especially during development, is when an asynchronous method never calls back. It may be because a service never answers, or because there’s a bug in the code that causes the callback to never be reached. No matter what the cause is, when that happens, you’re typically out of luck because by the time you notice the unusual delay, the execution engine has already left your code without leaving a trace. Breaking into the debugger will just lead you nowhere. You’ll have to tediously re-run and trace the execution of the code with lots of breakpoint into lots of callback functions. No fun.
-
It’s asynchronous all the way down
Captain Obvious has something to tell us:Asynchronous methods can call synchronous ones, but synchronous methods can’t call asynchronous methods, or they must become asynchronous.
Duh.
-
Fluent asynchronous API 6: recursive calls
This can never work:
api
.then(function thenRegisterNextTick(next) {
process.nextTick(function onTick() {
api.finally(next);
});
})
.finally(done);The “next” callback function gets called from a promise the execution of which depends on the completion of the “then” call. This is a deadlock, albeit a subtle one.
-
The mysteries of Orchard’s OEmbedPart, or how to integrate YouTube videos
The media library picker field does a fine job of integrating and rendering external videos such as YouTube videos. If you want to customize the rendering of the videos, however, you’ll have to deal with the OEmbedPart (obtainable from contentItem.As<OEmbedPart>()). OEmbed is a fairly relaxed protocol that is designed to enable the integration of bits of a site into another. It can deal with images, videos, or even arbitrary HTML. Because it is so vague in what it can do, it’s rather hard to give it a representation other than something completely dynamic, and that's exactly what OEmbedPart is doing. Its only structured property is Source, which is the URL of the original content, one of the only constants in the data. The rest is available through the part’s indexer, by name: part[“title”] for example.
-
Fluent asynchronous API 5: internally calling methods
One more thing about fluent asynchronous APIs… Because synchronous methods are changed into asynchronous ones by the flasync helper, internal calls of a synchronous method would in principle also have to be asynchronous calls. That would go against the requirements that I’ve set in the first post to provide a way for API authors to write code that is as close as possible to what they would write in a regular API.
-
Fluent asynchronous API 4: Nope, Q won’t work
As I said at the end of the previous post, I really thought that a promise library, Q in particular, would help, and would enable me to get rid of my helper library. It turns out, after I tried to make it work, that it won’t. The reasons have to do , among other things, with how hard it is to chain a new asynchronous method onto an existing promise, a strange limitation of those libraries. Well, in fact you can very well create a new promise that combines the old one and a new one for the new asynchronous operation, but that would mean managing a list of promises instead of managing the todo list of callbacks I have in my own helper. In other words, I would still have to maintain a helper that would be, maybe, marginally simpler than what I have now, except that it would require a dependency on another much more complex library that I didn’t write.
-
Fluent asynchronous API 3: the helper library
In this third post, I’ll show the inner workings of the little helper library that I wrote to build fluent asynchronous APIs. As you’ll see, it’s not simple, but analyzing it does, I hope, give interesting insights about asynchronous programming.
-
Fluent asynchronous API 2: building an API
In the first post of this series, I established my requirements for a fluent asynchronous API, and showed what using such an API looks like. In this post, I’ll show what the code for the API itself looks like. On this side of things, the requirements are that writing synchronous methods in such an asynchronous API should look as close as possible to writing the same methods in a synchronous API.
-
Fluent asynchronous API 1: API requirements
Today I built a fluent asynchronous API. This is hard to do if you want the API to remain as close as possible to a synchronous API, but still maintain reliable and consistent asynchronous behavior. There are examples of this around (Erik Meijer’s reactive extensions come to mind), quite a few –not always clear– blog posts, a general functional pattern, and a few npm packages. None of this however seemed to correspond exactly to what I wanted.
-
Dude, where are my images?
So you’ve copied some images somewhere in Orchard’s media folder, but they are not showing… What’s going on? If no media are showing on the site, chances are you’re missing a web.config file under the media folder. Put one in there. If the images are only missing from the Media section of the admin, then the problem is that you’ve created orphan media.
-
Some asynchronous JavaScript weirdness
I feel a little silly about the time it took me to find that bug, but it’s a fairly typical one, so it’s worth sharing. While writing a test case for an asynchronous operation, execution of my code would seemingly magically skip over a whole bit of code and jump directly to the assertions at the end of my test case. It looked really bizarre, as if JavaScript didn’t want to run my test code, and I suspected a bug in the debugger for a brief moment. Of course, that was just me not spotting the obvious bug I had written a few minutes earlier. But let me show you some simplified code that demonstrates the issue...
-
Exception Catch 22
That one gave me a really hard time, so I have to share… I’m implementing a logging module in a Node application, that uses Winston. The Winston feature that I wanted to test is the exception logging. That proved to be surprisingly difficult, because of conflicting exception handling from Winston and from the test runner.
-
Giving a few thanks
It has been my experience that the most rewarding communities to work with are open source communities. When you give your time and expertise freely, not everyone may recognize the value that you’re bringing to the table, but enough do. There is no better way for a developer to build a reputation, which eventually converts into more business, and better positions and wages.
-
Grunt in a modular application
As a Node application grows, a modular design where features reside in neatly separated units is a must in order to keep things manageable. Node makes this relatively easy with its package management. For the application as a whole, however, there is little guidance available to help you keep things consistent.
-
Use grunt.config.merge
Most code samples that you’ll find, both on the Grunt web site, and on the readme.md for Grunt plug-ins, will tell you to set the configuration using the grunt.initConfig method:
-
Tagging a fake Orchard content item
In my series of posts about building fake Orchard content items for testing purposes, here’s a short one that shows how to add tags to a fake content item. This one is interesting because it shows a basic case of relationship (between the item and its tags). The way tags have been implemented (it’s one of the oldest modules in Orchard, and one that should honestly be replaced with taxonomies in almost all cases), in order to add tags, we’ll need to create records for each:
-
Crockford’s 2014 object creation pattern
Douglas Crockford has long advocated for relatively unusual object creation patterns that do away with the “new”, and now the “this” keywords altogether. While watching a recent talk that he gave about the better parts of EcmaScript 6, I spotted the evolution of his pattern for the new version of the language. I haven’t found it explained anywhere else so far, so I thought I’d try to deconstruct it in a post…
-
Can we make commas optional in JavaScript literals?
Sometimes, small improvements can go a long way to making a language more enjoyable. One thing that I seem to feel acutely for some reason is noise. Noise is those parts of the language that –with compiler changes– you could remove without changing the meaning of the program, and without making it less clear. In many cases, removing the noise would actually make the code clearer, because the reader can focus on the meaningful parts without being distracted by the noise.
-
Adding fields to a fake Orchard content item
In previous posts, I’ve shown how to build fake content items, and how to use a fake content manager for testing Orchard modules. In this post, I’ll show how to add fields to a fake content item.
-
Stubbing the Orchard content manager
I’ve shown in the previous post how to build fake content items for testing purposes. When the code being tested gets content items from the content manager, however, you will also need a stub for the content manager, so your code receives fake content items that you have prepared, and not real ones from the database.
-
Faking Orchard content items for testing
When testing Orchard modules, it’s often necessary to build fake content items that will be usable by your code, but won’t be database-bound. For this purpose, I’ve built a number of stubs and helpers over the years that enable most scenarios to work using fake content items. Because everything in Orchard is based off interfaces and dependency injection, mocking is rarely necessary, and a few good stubs are often all you need.
-
Unit tests are to testing…
Developing good software at a large scale requires the collaboration of several disciplines, that are not, contrary to your boss’ opinion, interchangeable. You need developers, of course, but you also need designers, PMs, QA, writers, usability people, localization, pointy-haired bosses, etc. There has been a tendency in some companies, however, to try to get developers to do more and more beyond coding, or even to magically transform test engineers into developers.
-
Node pitfalls 3: this is not the file you want
In Node, each module has its own copy of its dependencies, under its own node_modules directory. Each of those modules in turn can have its own dependencies under its own node_modules, and so on, in matryoshka doll fashion. This is not a bad way to deal with dll hell: module A can depend on a specific version of module B, and cohabitate within the same application with other modules that depend on another version of module B.
-
ASP.NET v.next’s killer features are already here
ASP.NET v.next is definitely an interesting beast. It’s open-source, developed in the open, and the team has clearly studied the competition, Node.js in particular, very closely. Its killer features, however, the ones that may be hardest to reproduce by others, are not new features: we already have them.
-
Node pitfalls 2: lingering responses
One of the main appeals of Node is how it encourages asynchronous code. You may very well write synchronous code in Node, but if you do, you’re not going to reap the benefits, and you should probably use another platform. I’ll talk in another post about some of the challenges associated with JavaScript asynchronous code, but there is one specific consequence that made me trip more than a couple of times while I was learning Node: it is very easy to create wrong expectations about when or whether a specific callback will get called, and then things can go really bad.
-
My TechEd Europe talk on Orchard is online
I spent a lot of time on planes last week (which explains why I haven’t posted anything), on my way to Barcelona, where Microsoft had invited me to talk on the .NET Open Source Showcase. It was a great experience and opportunity, and I hope I did justice to Orchard in the time I had to present it. I focused on what makes Orchard a success story, and how to reproduce that success on other open source projects.
-
Fun with word mincing: what’s global?
I had a fun discussion with Rob on my last post, which led me to think about global variables and what we mean by “global”. Rob was arguing that a Node module is not strictly speaking global because it isn’t accessible unless you require it. That’s accurate, of course, but I’d still call that global in the sense that any code in the application that calls require will get the exact same object. The trick however is to only export from your modules constructors or factories. This way, the exported object may be global, but only in the same sense that a .NET class is a global object: everybody sees the same System.String, but it doesn’t matter in the least, because the string instances that you create from it are not global unless you make them. It’s still a little easier to shoot yourself in the foot in Node because functions and class constructors are ordinary objects, and the wrong pattern of exposing a singleton as a module is quite easy to write, but I’ll admit that it’s a mistake you’re probably going to make only once. So. On the same page here, I think.
-
Some Node pitfalls – 1. Global state
I’ve been teaching myself Node over the last few weeks, by building a non-trivial application. In the process, I’ve been faced with a number of difficulties that I think are worth blogging about. First, let me say that I really like Node. It’s fast, powerful, smartly-designed, and has a liberating effect on my programming style. It is definitely trying to steer developers to a pit of success. It tends to do so, however, by surrounding the pit of failure with deep holes with spikes on the bottom…
-
For the love of OCD, show whitespace in your IDE
It’s a simple thing, and it will make it immediately obvious when one of your files contains accidental indentation tabs instead of the spaces that should replace them, or trailing spaces. All IDE and code editors have an option to show whitespace. I always have it enabled. The subtle glyphs that will materialize the spaces and tabs are hardly noticeable while you’re working, except when something unusual is where it shouldn’t be:
-
Identity in Orchard Import/Export
Orchard has a really neat concept of identity that’s mainly used when importing contents into the CMS. One of the difficulties with importing contents is that you need to make sure that you can import not just new items, but also updates to existing items. For this to work consistently, we need to be able to identify a content item reliably.
-
Opting out of anti-forgery validation in Orchard
Anti-forgery tokens are a very important security feature of ASP.NET MVC and Orchard. Most of the time, you should keep them in place, and just let the system work its magic. There are a few rare situations however where it’s not the appropriate protection and you’ll want to disable it. Being too lazy to include the token in your ajax requests or your forms is of course not one of those situations.
-
Reducing coupling with dynamic languages
I’m learning Node currently, after years of doing ASP.NET MVC, and a bit of Python on a couple of projects. There are lots of habits to shake off, and there are things that I miss (such as ASP’s outstanding model binding), but there is also a very liberating power in JavaScript, that lets you do things in a much more straightforward and even cleaner way than you would otherwise. There’s a lot less ceremony, and you can focus on what counts. One thing that keeps astonishing me is how I can make my Node modules work together without coupling them.
-
WebAPI actions in Orchard
Building WebAPI controllers in Orchard is fairly simple: just inherit from System.Web.Http.ApiController. You’ll then be able to inject dependencies exactly in the same way that you would anywhere in Orchard. WebAPI is designed so that the default behavior is that a controller represents a category of resources, such as a product, an article, etc. There’s a bunch of conventions in place so that just naming the methods on the controllers is enough to wire them up. If this REST-like behavior is what you’re after, that’s great, just apply the conventions and you’re good to go. If you need to stray from that model, and implement something closer to what you’d do with a regular MVC controller, you’ll need to do a little more work.
-
Testing Node.js code that requires JSON files
A preferred way of creating a JavaScript object from a JSON file is to use the require function. Require will take care of the file’s encoding, and will cache the results so reading the same file a second time will not hit the file system. Testing such code can seem challenging, however.
-
Some challenges with Node.js on Windows
While there are a couple of really good Node.js IDEs (I use WebStorm), developing for Node on Windows is challenging. The platform is clearly built for Unix-type systems, and Windows support is a lagging afterthought. If your dev machine is running Windows, and you want to develop for Node, you’ll need to be aware of a number of things.
-
Building a WebAPI route in Orchard
There’s a number of differences between regular MVC controllers and WebAPI controllers that make the latter much more suitable to building APIs: they are REST-centric, they can negotiate the format of the answer, etc. Implementing WebAPI controllers in Orchard is really simple. First, you need a route, and that will be the subject of this post.
-
Cleanly getting a WebAPI action URL
Whenever you need to get the URL of a ASP.NET MVC action, you should use Url.Action (where URL is an instance of UrlHelper), and never hard-code the URL. This way, the URL is dynamically constructed from the available information in the Url.Action parameters and in the route table. If the route is changed, the results of the Url.Action call will change accordingly, and everything will continue to work. The same principles, of course, apply to WebAPI actions.
-
Adding dependencies that don’t implement IDependency to Orchard
There are rare cases where you’ll want to be able to inject instances of classes that don’t implement IDependency. One example of this can be found in MvcModule in Orchard.Framework, which also provides a good example of how to do it. The idea is to derive from Module, and override the Load method to register factories for the types you want to expose:
-
Making MiniProfiler work in the Orchard dashboard
MiniProfiler is a wonderful module. It’s especially good at showing you the select n+1 problems in your Orchard applications. For some reason that is not entirely clear to me, however, it is only active on the front-end. If you have to debug a performance issue in the dashboard, you’re out of luck. Fortunately, the limitation is entirely arbitrary, easy to find, and easy to remove.
-
A better way to play with HQL in Orchard
In previous posts, I’ve shown how to query Orchard with straight HQL. However, I haven’t provided a good environment to run and debug these queries so far, because I didn’t have one. As a matter of facts, my method to build new queries has consisted in building queries from the projection module, putting a breakpoint at the end of the “DefaultHqlQuery.ToHql” method, and previewing the query in order to steal the query string built by the projection module. To debug, I’ve put the code in custom controller actions, and debugged the exceptions from VS. Not super-convenient.
-
Querying Orchard fields with HQL
Before projections, the official word on fields was that they weren’t for querying (they are stored in an XML blob on the content item record). Projections enabled field querying for the first time, through special tables that index field contents: Orchard_Projections_DecimalFieldIndexRecord, Orchard_Projections_DoubleFieldIndexRecord, Orchard_Projections_IntegerFieldIndexRecord, and Orchard_Projections_StringFieldIndexRecord. Each table is specialized for one underlying value type.
-
A storm of fail
It’s OK to not know something. It really is, if you’re willing to admit it, and have a reasonably accurate idea about your own level of competence. Unfortunately, thanks to the Dunning-Kruger effect, we’re all better than average drivers, parents, and… security experts. Through the excellent @InfoSecInsanity, I got pointed to a veritable (and unfortunately, involuntary) repository of ways you can screw up your web site’s security. I don’t want to pick too much on the author, as as far as I know, he’s well-intentioned, but he should really retract his post, as well as any other post he wrote about security, and take a few courses. He might also want to stop calling himself an”expert” about topics on which he has clearly no expertise. Writing crappy code is one thing. Propagating dangerous code through blog posts, that hundreds of clueless people will copy into their own applications, is another.
-
Getting Orchard content items out of HQL
I the two previous posts (here and here), I’ve showed how to build HQL queries against the Orchard database. Once you’ve built the query, you’ll want to get results, often in the form of fully-built content items. In lots of cases, you’ll want to paginate the results, for which you’ll need a total count, and detailed results for only the current page. This post will show you how to do all these things.
-
Joining Orchard part records in HQL
In yesterday’s post, I showed the basics of HQL querying in Orchard. It is by far the most flexible way to query Orchard’s database, but one thing I didn’t show is how this works in relation to Orchard’s content type system. Querying over records is nice, but if those records are part records, you need to be really careful and check that they correspond to a real content item, that his content hasn’t been archived, and that its publication state is what you need it to be. In order to do that, you’ll have to join with ContentItemRecord and ContentItemVersionRecord. But how do you express joins in HQL in a way that works with Orchard records?
-
Querying Orchard in HQL
Orchard has two APIs on IContentManager to query content items: Query, and HqlQuery. Query is the older API, but it’s also the simplest. It’s great when you want to perform a simple query such as “get all content items with part TitlePart where the title begins with A”. HqlQuery is a little more advanced, closer to nHibernate APIs, and allows for more complex queries. It was added to build the dynamic queries necessary for Projections. It is still, however, designed around the Orchard content type system, which makes it inadequate for queries that don’t trivially map to content items and parts.
-
Referencing server-generated fields in script
I had to fix a bug today in somebody else’s code that was caused by some Javascript referencing an element by id. The problem was that this id was assumed by the script to be “#Parameter1” whereas the server-side code generating it was:
-
A blogging experiment
I consider blogging to be a very efficient way to help the community while promoting my own personal brand (which is important not for my oversized ego, but because I’m an independent consultant, and more visibility means more and better contracts). Many successful bloggers blog often, sometimes several times a day. I’ve never been able to sustain a frequency of more than a post per week myself so far, but for about a week, I’ve been trying something different. I’ve decided that I would write something every day, whether it’s a post for one of my blogs, a page for a book, training material, or some personal notes. The idea is to make writing a part of my everyday routine.
-
Speeding up pages with lots of media content items
For a few versions now, Orchard has been treating media (images, videos, etc.) as content items. There is a special kind of field, MediaLibraryPickerField, that enables you to add images, or collections of images, to any content type. For example, on the Nwazet web site, I’ve added a “Image Gallery” field to the product content type:
-
Deriving from a fluent class
Here’s an interesting one, and maybe you can help me make my design less crappy. I have this library that I’m a little proud of, called FluentPath. It’s a fluent wrapper around System.IO that enables you to manipulate files not unlike how jQuery manipulates the DOM, with operations over sets, and lots of functional concepts:
-
Strongly-typed HTML helpers in Orchard shapes
Orchard uses dynamic objects called shapes as view models. It happens commonly that you’ll want to use strongly-typed HTML helpers such as Html.TextBoxFor from within a shape’s template. Unfortunately, this is not possible: those helpers rely fundamentally on the model being strongly-typed, because they need to be able to infer the properties of the object from the Lambda expression passed as a parameter of the helper, but the compiler will refuse to compile Lambda expressions on dynamic objects. As I understand it, this is because the compiler can’t infer enough information about the object at compile time. This is all regrettable but that’s how things are.
-
How to shoot yourself in the foot
Here’s another one to throw onto the “how to ask questions” heap. Sometimes, somebody asks a question on Stack Overflow that amounts to this:
What is the best way to shoot myself in the foot?
-
On accessing chains of potentially null properties
Raise your hand if you’ve ever written code looking like this:
-
Asking questions is a skill
If you’re going to get into any sort of technical job, you’re going to have to ask questions. A lot of questions. Unfortunately, too few people understand how to ask questions properly. Asking questions is a skill. It has to be learnt.
-
The CMS ecosystem and Microsoft
CMS is extremely important strategically for any web company. About 35% of web sites use a CMS, and the top ones are all PHP (WordPress on its own is more than 20% of all web sites). In other words, if you care about the market share of your web platform, you need a good CMS running on it.