Feature Team: Autoroute

Topics: Announcements
Coordinator
Oct 5, 2011 at 10:36 PM

This thread is for the Autoroute feature team. If you want to help with the design, implementation, testing and documentation of this feature, please chime in here.

The Autoroute feature's goal is to simplify and improve the handling of friendly URLs in Orchard. The feature is explained in this post: http://weblogs.asp.net/bleroy/archive/2011/07/30/future-orchard-part-3-autoroute.aspx

Initial members:

  • Sébastien Ros
  • Bertrand Le Roy
Oct 6, 2011 at 7:23 PM

Hi all,

Just had a first look at the Autoroute plan and I have to say it looks astounding. I've spent a quite some time trying to build a system that automatically creates well-structured Urls based on connections between items (in a flexible and extensible way of course), whereas Autoroute just looks way more straightforward whilst giving more control - profoundly simple in fact.

I thought it might be helpful to just share some particular goals I've been trying to achieve which have been particularly tricky to get right (currently my system is working for a lot of cases, but still not all). These are things I didn't see explicitly mentioned in the blog post, but which for me would cover pretty much every routing scenario I've ever faced; so I'm just wondering if members of the team have thought about these aspects and to what extent Autoroute might already be going to support these features. I'd like to be considered to help with the implementation - to me it's a really important area to get right, which is why it's something I've been spending a lot of time thinking about.

Recursive Urls

On the blog post the common example was of the /my-container/my-item variety. In practise it's nice to have much deeper levels of Url hierarchy; e.g. /category/category/category/product (where categories are all the same content type but in a hierarchy). I'm sure Sebastien will be considering this for  Taxonomies!

Items on multiple Urls

Of course it's bad SEO to repeat the same information on multiple pages, but here I'm talking about showing different information a common object. For instance, different tabs on a product information page: /products/foo, /products/foo/details, /products/foo/specifications, /products/foo/reviews - so we still want to show the Product heading, price and photo - but show a different set of information on each tab. Will the Autoroute be able to support this kind of structure?

Filter Urls

To take another example from the blog post: /my-blog/2011/5/15/my-item

If I saw a Url like that in my address bar, I'd expect that I can delete /my-item from the end and see all blogs from the 15th ... then delete the /15 and see all posts from May ... and so on. So ideally there would be a standard way to tie the route patterns into aggregate pages like this.

Moved Urls

One thing I've wanted to do for a while is a way to keep old Urls alive (with a 301 Moved Permanently redirect). It's so easy to leave inbound links broken when you restructure a site - and I run into this all the time on internet at large, it can be really frustrating. It'd be great to just store a table of Url redirects, and have this updated any time a Url changes.

Coordinator
Oct 6, 2011 at 7:33 PM

Pete: if you want to be part of the feature team, please create an account on Trello and send me e-mail so I can arrange for your contributor agreement with Outercurve.

The filter Urls point may be tying into the Projector feature.

For moved urls, there is the rewrite module, but yes, the alias table should have the notion of canonical URL and permanent redirect, somehow.

Oct 19, 2011 at 1:11 PM

What's the next step with this feature? I see the Trello task "Autoroute: backport from 2.0 to 1.x branch" - does this imply that some work has already been done, or is that for after the feature is implemented?

Coordinator
Oct 19, 2011 at 11:44 PM

A lot of work has been done in the 2.0 branch, yes. Next step would be to bring that code into 1.x. well, in a fork, of course. If you want to get that started, please ask Sébastien to create the CodePlex project and the fork.

Oct 20, 2011 at 4:25 PM

Ok - Sebastien, can you do that? I'm kind of thinking it could be easier for me to tackle this than to get everything resolved with my own routing system (although I have it basically working, there are various aspects I'm not quite happy with how they've turned out!)

Oct 20, 2011 at 4:45 PM
bertrandleroy wrote:

A lot of work has been done in the 2.0 branch, yes. Next step would be to bring that code into 1.x. well, in a fork, of course. If you want to get that started, please ask Sébastien to create the CodePlex project and the fork.

Where is this branch located?  I see no 2.0 branch in the main repo, and see nothing about autoroute in any of the commits.  I'd like to take a look at the autoroute stuff.

Luke

Oct 20, 2011 at 4:53 PM
ldhertert wrote:
bertrandleroy wrote:

A lot of work has been done in the 2.0 branch, yes. Next step would be to bring that code into 1.x. well, in a fork, of course. If you want to get that started, please ask Sébastien to create the CodePlex project and the fork.

Where is this branch located?  I see no 2.0 branch in the main repo, and see nothing about autoroute in any of the commits.  I'd like to take a look at the autoroute stuff.

Luke

I was wondering the same thing, I thought perhaps Bertrand might have meant just the dev branch, but couldn't see anything relevant there.

Coordinator
Oct 20, 2011 at 4:53 PM

This is the dev branch. Maybe it should be renamed 2.0

You can't find anything about "Autoroute" right now, but Alias module, and look at the RoutablePart changes. Committer is Dave, if it can help you filtering, and Louis for Alias.

Oct 21, 2011 at 4:59 AM

So I poked around the alias source, haven't run it to see how it looks/works yet.  Very interesting code...but I'm curious about something.  I didn't really see significant changes in the main orchard repo regarding RoutablePart, would it be in a fork?  Honestly, from the brief look at the Alias code, it seems like it works almost independently of the RoutablePart and I don't really see from looking at the code how it can be at a point where it is approaching being a viable alternative/upgrade/whatever to the RoutablePart.  

I guess I'm saying that I'd be willing to help backport this functionality, as I'm sick of fighting with the current Routable limitations, but I am wondering if it's even at a workable state right now.

Oct 21, 2011 at 6:19 AM
Edited Oct 21, 2011 at 6:20 AM

I had a look around as well. There aren't any changes to RoutePart in dev - however there is a commit that references another project - http://orchardroutable.codeplex.com/ - which purports to be Routable v2. It does hook into the Alias module but there isn't any usage of the tokenised functionality as described in Bertrand's blog. The changes in dev are mainly related to removing dependencies on IRoutableAspect (which was previously renamed to IRoutePart in dev) and removing Containable's own route constraint, intending it to be replaced by the new routing. From what I've seen I'm not quite sure if the new routable is working even for normal pages, let alone containers. Sebastien, is it possible to run these findings past Dave/Louis and check we're looking at the right/latest versions of these projects and not missing anything, and also any other information and guidance they might have? A big problem with what I'm seeing is there are practically no comments in the code, making it very hard to work out the intent of some sections of it!

Coordinator
Oct 21, 2011 at 3:58 PM

You should have come to those design meetings, would be simpler now ;)

Coordinator
Oct 21, 2011 at 8:14 PM

Dave sent me his brain dump from what he had done for 2.0:

Alias

Lives in orchardalias.codeplex.com. Louis wrote it, pretty much, but I ran into some bugs when I wrote the new Routable part, and made some non-trivial changes. There are likely some more bugs. One known issue is that the in-memory copy of the aliases are not updated immediately upon CRUD’ing an alias via the service interface – it takes until the background task picks it up. Incomplete: Louis was considering ways to expose this service in Core without pulling the entire module into Core, but that’s all the context I have. This needs to be considered so that modules can CRUD aliases without having a horizontal module dependency on Alias.

The Old Core/Routable

I was in the middle of phasing out this module by removing all the dependencies we have on it in core and framework. There are a few strings left to cut, but I got stuck on these as they require some design choices and we never really sat down to figure them out (full context/list of issues with this in the thread below). Once all these strings are cut (if they can be), the Core\Routable module can be deleted, and the IRoutePart interface in framework\ContentManagement can be deleted. Oh, and IHomePageProvider and the Core\HomePage module can be deleted, too.

The new Routable

Lives in orchardroutable.codeplex.com. It’s a very simple part, beautifully so, because unlike all the complexities of the old one, it doesn’t care about containers, doesn’t separate slug/path, and doesn’t use IHomePageProvider. All it does is use the Alias service to maintain the alias for a content item upon saving, and removing it upon removal (just think of it as an automatic way to create and maintain the alias for a content item – something you could do by yourself by using the Alias UI). Aliases are not versioned by design (drupal). It probably needs some proving for other odd scenarios like what happens, or should happen, to the alias when you unpublish… or how it should handle conflicts with existing routes (it currently would just overwrite the old one, I think). Also, currently the record contains Title, and Bertrand and I are in agreement that Title should be separated (see more context on that way below), in which case the Routable part actually will no record data at all.

Note from Bertrand: in 1.3, we actually created an additional interface for title, and implemented that on Routable, which is less upsetting for existing contents and works just as well, as other non routable such as widgets can implement just that to have a title.

Containers

I have removed Container’s reliance on Core\Routable, but this puts them into a transitional state you should know about. Previously, Container had an IRouteConstraint provider that effectively hijacked routes to containers in order to force them to its own controller action. That controller action then used the incoming path to figure out which container to display. Well, now that we have Alias, it should in general be unnecessary for modules to hijack routes in this way. Instead of using a route constraint to force routes to its own controller, it simply needs to modify the DisplayRouteValues of the content item, and the new Routable part will create an alias using those route values. Another advantage of this technique is the route can be smarter and contain data – now the Display action for a container is given the id of the container directly, rather than it having to figure out which content item it is from the incoming path. This is all done already. However, that means containers are kind of broken by default in dev – to fix, you must remove the core Routable module and add the new Routable module, because otherwise, no Alias is setup that points to the container’s controller, and there’s no route constraint provider anymore, so the route to the container is handled still by the old Routable controller instead of the Container’s controller, which will just render it incorrectly.

Orchard.Blogs

Lots to do here. Blogs relies on core\routable in a lot of subtle ways I was only beginning to understand. Rather than rat nest into all the issues, we decided to just remove Orchard.Blogs from the solution for the time being (as Louis put it, we didn’t want to design Routable around Blogs). See, like containers, Blogs forces routes to its own controller (I call it ‘route hijacking’) using a IRouteConstraint. That should be unnecessary for the same reason it is now unnecessary for Containers – routes, thanks to Alias, can now point at any controller action, not just content items, so hijacking a route is just a matter of changing the RouteValues that are assigned to the content item’s alias (and that is done by setting its DisplayRouteValues in metadata). But to complicate the issue, the Archives feature provides some routes on top of the blog routes, and in my 1st impression this seemed like it would be difficult to rework with the new system (and it could possibly still require route hijacking).

IHomePageProvider

The old routable module uses this – the new one does not need it. The homepage is set simply by setting an empty alias, so this abstraction is no longer necessary. It should be removed when the core routable can be removed. However, Bertrand had some ideas about better management of the site home page – basically, that it should be done from a site level setting, not per-content item.

Note from Bertrand: yes, the home page should be in site settings, even if you can still set it from the item editor. The work we're doing with content reference field may yield a good content item picker that we could reuse in site settings for this. but of course, setting the alias to / on any content should work too, although we'd better have good UI to handle conflicts there.

Data Migration

We need to be able to provide a migration that converts core\routable data to the new 2.0 modules. How to do this is unknown/undesigned.

Autoroute

0%

Oct 21, 2011 at 8:55 PM

Wow, thanks for all that info.  The only immediate feedback I have is to wonder why there is an attempt to keep alias out of core, especially if routable is dependent on it?  Also, are there other usages for alias that you guys have considered that would merit it being a completely separate module?  I understand how it fits into the routable scenario, but I'm trying to think of another use case for it that would warrant it being a standalone module and can't come up with anything.

Coordinator
Oct 21, 2011 at 10:25 PM

Well, it's more like a general philosophy that we've been following from the start: we only put it in core or framework if we think no site will be able to function without it. Alias is borderline here, but one could very well imagine a custom Orchard site that is so locked down that all contents are served from a custom controller that responds to a custom route system that is based on some completely different rules. Actually, you might be able if you want to, to resurrect the current Routable into a 1.4 instance of Orchard. At least in principle. Even if there were no practical appliations of separating it, it's a good design rule of thumb because it encourages you to factor things in a way that avoids any unnecessary coupling.

Oct 26, 2011 at 4:23 PM

Thanks for the detailed information - this closely matches my findings, and adds a lot of extra context.

It seems the overall strategy for this feature would be:

a) Test Orchard.Alias against 1.3. It might already be working, but I think there are a couple of issues, and it's lacking a test suite.

b) Get new Routable working against 1.3 and with Alias. Hopefully not too major.

c) Implement Autoroute!

Let me just go over the issues I think there are with Alias. What I'm noticing currently in the code is that it's still falling into two of the pitfalls of the old routing system:

- It requires reading an entire database table into memory, not only on application startup but also every time the in-memory alias map gets updated (in a background process). This is still a much better situation than old routable, which reads all routable content items from the database (clearly a more expensive operation due to joins with part records) - but it's still not optimal and with a really huge alias table this will surely cause a big delay on application startup.

- More importantly, it's using a plain lock (i.e. lock(_stateLock)) to handle thread-safety when both reading and updating the in-memory alias map. This means that for the entire time the alias map is getting updated, zero requests can be performed (even if that request was for, say, an ordinary controller!) ...And if multiple requests are hitting the server simultaneously you could also see problems.

Hopefully it's clear that neither of these two behaviours are desirable if we want a remotely scalable routing system! I think it'd be good to think about getting this system right first before building out the other features. I'd be interested in any input on the most performant and scalable way of achieving this. My own ideas are as follows:

1. Personally, I think it's better to read the routes directly from the database, rather than holding the entire list in-memory. On a site with millions of pages you can see how it would be a problem. With Alias we only have to read a single record from the database to get the route information, and reading a single record is not a very expensive operation. With the old system the benefits were clearer, since you had to query the content items database which is more expensive. Of course if there is data to show that it's significantly better to avoid hitting the database during routing then there is a third way - each time there's a request, check the alias map, and check the database only if that Url isn't known yet. This way the route table is gradually cached and you avoid ever having to read the entire table at once. Now, there is a technical problem in all of this - I actually already tried to write a very similar system (don't get me wrong, Alias is way better in that if maps to controller actions, whereas mine simply mapped to content Ids). The problem I ran into was that dependency injection seemed quite broken at the time my route constraint was instantiated; I couldn't get a functioning WorkContext or Work<T> (which I would then have used to get hold of an IRepository<TRecord>. I'm not sure where this problem lies in Orchard or how to overcome it, or even if it's changed in 1.3. I'll have to experiment again and provide some more info.

2. For the locking strategy, I would opt for either a ReaderWriterLock or the Monitor pattern (this was suggested in an older thread on this topic). This will ensure that for normal page requests (i.e. read operation) there's no locking; locking only occurs when the map is being updated. With 1) above, map updates are only ever a single entry at a time.

3. Finally, Alias is actually lacking something that the old routable had - the ability to update a single entry of the in-memory map. Alias just updates the entire map at once every time the background task runs. Presumably this needs to happen every time a content item is created or edited, otherwise routes would be missing! Old routable (and my own routing system) just update the entries for the item(s) that have changed when they're edited. Again the system described in 1) would negate this problem.

Oct 26, 2011 at 4:29 PM

I meant to add:

With regards to my point earlier about setting up redirects for moved Urls, this would be very easy to implement within Alias and I think it'd be a very nice feature to have. It'd be as simple as providing a redirecting controller and providing an extra Alias function to support it (could easily be a separate feature, but if it's available I'd be inclined to support it in the new routable!) If the feature is approved of I'd very much like to include it.

Oct 26, 2011 at 5:31 PM

Good feedback Pete.  A couple points:

  1. I've run into this conundrum before with in-memory vs database calls for permalinks.  It's really difficult because there's no one-size-fits-all solution.  For people with a smaller site, it absolutely makes sense to keep these items in memory.  For people with millions of URL's, that obviously isn't feasible.  The solution I've utilized in the past is to make heavy use of caching, but it needs to be finely configurable.  The first consideration is that these items should have a higher priority in the cache, because they are (usually) the most frequently accessed items on a site.  Secondly, you need some way to prioritize what items should be cached.  It could be as simple as using a sliding expiration, or something along those lines, or keeping track of a hit count and giving higher priority to urls that are frequently accessed.  
  2. One thing that could really come into play here is that (I think) we're assuming that every item that has an alias needs to be stored independently.  This very well could be the case, however when we start talking about the tokens/auto-route stuff, then we get to the point where we can have generalized paramaterized route (a la the standard mvc routing) that encompasses the majority of items on a site.  I'm not sure if this is clear, and I might even be way off in my assessment of the current status, however I think this would significantly cut down on the overhead in most cases.  Just to clarify - Say we have a blog post route rule (using tokens) that looks something like this: "/{blogId}/posts/{blogPostSlug}". My point here is that unless this route is manually overridden, I don't really see why an alias would need to be persisted/queried for any blog post that follows this structure.  Obviously you need to be able to a) Determine the permalink for the blog post and b) Route a request from that permalink to the appropriate controller/action.   Those are both feasible without persisting the aliases individually.  The biggest issue that I can foresee is the lack of ability to check for conflicts. I can think of a handful of solutions, one of which is that perhaps the entire list of aliases can be persisted for actions like this that happen infrequently, however the alias list that is used for permalink resolution only contains the generic route rules + any permalinks that fall outside of those rules.
  3. On the issue of redirects, I had meant to bring it up, but I'm surprised that this is being considered as an afterthought.  While I think the functionality should be able to be disabled/overridden, I would think that the default desirable behavior would be that if a permalink is altered, the old permalink would redirect to the new one.  Looking at the wikipedia page, this apparently isn't part of the "definition" of a permalink, but it just *feels* like that should inherent in the "perma" part of permalink.  Maybe I'm way off on this.
  4. Just curious, but what are your thoughts about making the "slug" system available outside of routable?  At some point it just gets ridiculous, breaking stuff out further and further, however I've found myself several times wishing that I could just get the slug piece of routable (similar to how we saw the need to break out title).  I know I could implement this in a separate module, and I also know that many of the times I've wished for this were in cases that will be addressed by the new routable...but I just wanted to through it out there.

This looks great.  Too great actually.  It's making me cry inside as I fight with the current routable stuff on a project that really needs this new functionality.  I wish I had time to help make it happen for this project, but the timeline is too tight.  

Coordinator
Oct 26, 2011 at 10:15 PM
Edited Oct 26, 2011 at 10:30 PM
parameterized alias are out of the question, sorry. That is the whole point: you have a recipe for the alias (the AutoRoute) and the outcome of this one time process is a stable alias.
Oct 26, 2011 at 10:25 PM
Edited Oct 26, 2011 at 10:26 PM

Thanks for your input, I'll respond to your points as numbered:

  1. It's something I also thought about back when I was trying to build my own CMS :) I'm inclined to think that building an elaborate in-memory cache in order to solve a problem that can be very easily reduced to a database optimisation issue, is somewhat "using a mallet to crack a nutshell". Since you and the team are doing a very good job enabling the many layers of NHibernate caching, and since the table can be indexed anyway, I'm not sure if a custom cache would really improve things, and could just introduce extra overheads (and definitely complexity). In the very simple case of a small site with only a few pages, is there really any need for the in-memory optimisation, since a small and simple site will be running pretty quickly? The real question is: what does a one-record indexed query cost (on a table with half-a-dozen columns)? If the answer is that the cost is trivial, then what problem do we need to solve?

  2. Currently the way the Alias implementation and the Autoroute spec looks, is that each tokenised route would be fully expanded and then stored in the Alias table as a complete url. Again, tokenised content-bound routes is something I started trying to implement on my own CMS. In the end I determined it was actually more optimal just to store each Url in a table! Looking up a Url in a table requires exactly 1 query (and on a database column which can of course be unique and indexed, therefore extremely fast to look up). However, if you start processing tokenised routes, you might suddenly have to perform dozens of database queries in the route constraint and for every single page request.
    Let's use a blog post as an example: /blogs/{blogSlug}/posts/{year}/{month}/{day}/{blogPostSlug} - what do we have to do here to determine if we have a valid route? First, we have to query content to find out if there's a blog called {blogSlug}, and we have to look up its integer Id. Then, we have to query content again for a blog post that a) is in the correct blog and b) is in the time range as specified by the date. Now suppose we don't find it - that means we still have to check any other tokenised routes as well! If all tokenised routes fail, we can return false from the route constraint and fall back to the rest of MVC's route table.
    So the more modules you have installed, the more queries (as well as string processing and matching) you potentially have to run just to find out if the route constraint passes. Indeed, since you're querying all these items, you might as well pull the content out of the database at that time (rather than having to do it again later...) and then you start needing an efficient mechanism of exposing those items in whatever controller finally handles the request. All in all such a system adds a lot of complexity for a somewhat meagre saving of exactly one record lookup per request, whilst potentially adding a lot more processing and database querying for what could be a substantial proportion of requests.

  3. The redirect feature certainly isn't an afterthought for me - again it's something I was building into my previous CMS because I think it's really important that, as you say, permalinks are actually permanent. It's obviously just not something the Orchard team thought of until I mentioned it (at a fairly late stage in the whole game!) But, I'm not aware of any other CMS's or even web frameworks at large that support such a thing (of course there must be some, it's just not something I've seen), so I think this could be another great selling point for Orchard. It is extremely straightforward to implement within Alias and I see no reason why not to. The only possible reason I can think why someone might want it disabled is if they were trying to keep their Alias table small on a site where Urls changed all the time (even so, I can't think of a specific example for such a site). Of course, I'm not justgoing to go and include a feature that hasn't been specified or approved by the team, but even if they didn't I'd just include it in Mechanics and of course support it natively with the urls from my Plumbing module :)

  4. In effect, Alias is making the "slug" system available outside routable. You'll notice in Louis' comments above he states that the new RoutablePart will have no fields of its own, now that Title has been outsourced. This is because the new Routable just takes the title, converts it into a slug (or takes the user input), and stuffs that into the Alias table. Actually, the current routable is little more than a "SlugPart" anyway now that title is gone. With Alias you can assign any url to any set of controller/action/routevalues so really you can do anything you want with it.
ldhertert wrote:

This looks great.  Too great actually.  It's making me cry inside as I fight with the current routable stuff on a project that really needs this new functionality.  I wish I had time to help make it happen for this project, but the timeline is too tight.  

 

Well, I think I can quite quickly get Alias and Routable working (I could do with it for the site I'm building) and hopefully implement some strategy that makes it more optimal than current routing. Redirection should also be pretty quite and I'd also like that for current projects, Google has been getting horribly confused whilst I've been restructuring the site and changing Urls and it would have been great to have so far. So maybe some of this stuff isn't too far off; Autoroute itself isn't even too much to do on top of all the existing support of Alias and Tokens...

Edit: Bertrand stated reply to 2) somewhat more succinctly than me :)

Oct 26, 2011 at 10:32 PM
Edited Oct 26, 2011 at 10:34 PM

 

BertrandLeRoy wrote:
parameterized alias are out of the question, sorry. That is the whole point: you have a recipe for the alias (the AutoRoute) and the outcome of this one time process is a stable alias.

 

I don't think I really continued to follow through with the implications of what I was saying.  As I typed, my view on the subject changed.  It makes sense to have all of the aliases saved in the database for the things that it's really necessary to compare full paths for, like for checking against permalink conflicts.

What I was getting at is that I think we will run into issues very quickly with route resolution, for example, resolving the proper action/controller for a given url.  What I'm suggesting, and maybe this is already the plan, is that it doesn't make sense to have every possible alias stored in the routing table, when a majority if not all of those aliases are following a predictable structure.  So I guess what I'm suggesting is that while the default behavior of registering a route for every alias in the database makes sense from the alias level, if you've got autoroute sitting on top of it, I think it would make a lot of sense to override that behavior and be a little smarter when registering the routes.  Basically, it knows what the tokenized rules are, so when it loops through all of the aliases, it can look to see if they fit within the rules, and if so, you can cut significantly down on the routes. Does that make more sense?

Coordinator
Oct 26, 2011 at 10:34 PM

+1 on permanent redirect handling by alias.

Coordinator
Oct 26, 2011 at 10:37 PM
Edited Oct 26, 2011 at 10:38 PM

One of the behaviors we want is that if you change the recipe for autoroute, an existing item whose alias was generated using the old recipe won't change. Furthermore, a tokenized alias is a *route*. You already have that feature. It is just a route. You don't need alias for that, and it will continue to work. It is a distinct feature that we already have. This is not the problem we are trying to solve.

Oct 26, 2011 at 10:49 PM

Pete, just a few responses, again numbered:

  1. After thinking about it some more, I realized that I was off base by assuming that it was going to hit the database every time to resolve a route.  It will be looking at the routing table.  The question is, can we make the routing table more efficient.  On a secondary note, I just have to say that the "just one query" is a fine point in most cases, but when I profile Orchard it makes me somewhat sick to my stomach.  I understand why it is the way it is...there's really no great ways around it, but that's what drove me to work on the nhibernate caching.  That said, I feel like the core aspects of orchard need to be as optimized as possible when you consider how easy it is for custom modules to have hundreds of "just one query" situations.
  2. See my previous response.  Tokenized routes would not require database access on every request, As can be seen in every MVC app out there.  All it does is register a generic route that says, if it matches this pattern, go to this controller with this action.  It's up to the action method to determine if it is a valid route, which is done all over the place in MVC in general, as well as many places in Orchard.
  3. I wasn't suggesting that it was an afterthought for you, as you were the one suggesting it.  It just seemed like a natural fit for me in this scenario, and I was surprised that it wasn't included in the discussion until this point.
  4. I know what you're saying, however I was talking about making the slug system available to content types that aren't routable.  I don't have a great example, but I've run into it a few times where I wanted a unique string identifier for use inside the logic of my module.

I understand that none of these are super critical issues, and the tokenized routes really won't make a huge impact on the performance of the majority of sites.  I just thought it would be good to get it out there.

Oct 26, 2011 at 11:03 PM
ldhertert wrote:

Pete, just a few responses, again numbered:

  1. After thinking about it some more, I realized that I was off base by assuming that it was going to hit the database every time to resolve a route.  It will be looking at the routing table.  The question is, can we make the routing table more efficient.  On a secondary note, I just have to say that the "just one query" is a fine point in most cases, but when I profile Orchard it makes me somewhat sick to my stomach.  I understand why it is the way it is...there's really no great ways around it, but that's what drove me to work on the nhibernate caching.  That said, I feel like the core aspects of orchard need to be as optimized as possible when you consider how easy it is for custom modules to have hundreds of "just one query" situations.
  2. See my previous response.  Tokenized routes would not require database access on every request, As can be seen in every MVC app out there.  All it does is register a generic route that says, if it matches this pattern, go to this controller with this action.  It's up to the action method to determine if it is a valid route, which is done all over the place in MVC in general, as well as many places in Orchard.
  3. I wasn't suggesting that it was an afterthought for you, as you were the one suggesting it.  It just seemed like a natural fit for me in this scenario, and I was surprised that it wasn't included in the discussion until this point.
  4. I know what you're saying, however I was talking about making the slug system available to content types that aren't routable.  I don't have a great example, but I've run into it a few times where I wanted a unique string identifier for use inside the logic of my module.

I understand that none of these are super critical issues, and the tokenized routes really won't make a huge impact on the performance of the majority of sites.  I just thought it would be good to get it out there.

1. You misunderstand. Alias registers one route with the system routing table. That route checks against an in-memory dictionary of url strings. I am suggesting we should just directly query the database alias table instead of the in-memory dictionary. And the question is, can database indexing and NHibernate caching make that operation suitably efficient? As you say, Orchard hits the database a ridiculous amount of times in a request ... does one single record query make much difference? A big part of the Performance team's task is reducing the amount of database activity - I don't think anybody is going to be worried about a single indexed record, especially if it results in less database hits at startup.

2. Standard MVC tokenised routes are binding against in-memory data (controller names, action names) and therefore don't need to hit the DB - but as soon as your tokenised routes are bound against content, you necessarily have to query the database to validate the route. Even in a fairly boring MVC /orders/{orderId} route, you still have to perform a database query to check the order Id even exists. It's better to reduce all that querying to a single indexed table than have a number of different routing strategies all performing their own database lookup.

3. I realised what you were saying ;) I wasn't too surprised at its omission, like I said it's not something I've particularly seen in other frameworks, but I'm glad I brought it up!

4. I'm not totally sure what it is you need from the slug module - but are you aware of the Identity module, if you want string identifiers?

Oct 27, 2011 at 1:37 PM

I delved deeper into an area of Orchard that I'd previously been scared to try and comprehend, namely Orchard.Framework/Mvc/Routes/ShellRoute.cs. This is where a lot of the horse work happens of Orchard plugging into the Routing/HTTP pipeline.

The simple reason I'm not able to access the database during routing is that the WorkContext gets created after the called to RouteBase.GetRouteData(...) - so it's impossible to perform any work (e.g. resolving an IRepository<T>) at that stage.

To expand on the implications of this, there is no way for an Orchard route to bind to anything in the database - you can only check the database in a filter, or in the controller itself.

Looking further at the code, it's not immediately obvious how to resolve the situation. I wonder does anyone have good knowledge of routing and ShellRoute who could comment on whether it's possible to change this behaviour?

Coordinator
Oct 28, 2011 at 5:50 AM

Here's what Louis answered:

Yep, thing is routes can be registered from any active shell. Even with one site when you make a config change there's one shell retiring as a new shell is incarnating.

So - for each route to talk to its database through •its• hibernate lense... All complicated by the fact transaction scope is a thread-wide effect... It's just not good. Blows up in your face as soon as looks at you.

•Plus• each call to actionlink, every call that turns route values to a path, •also• rips through the routes match function to find the hit. So database calls in a route would destroy your performance while rendering an a few orders of magnitude more than you'd expect if you're only considering the db calls needed to match the incoming route.

Good news is there's a way to do this we used in the content route before. The route provider for the module gives the routes it makes a reference to a data-bearing singleton. An onchange and inthebackground task feed changing data into the singleton. The route can use the data in the singleton in it's algorithms.

He also implemented a route module that takes care of adhoc routes and the data pump for those adhoc routes. Wouldn't pathauto pump records into that route module, and let that module worry about getting a path to land on the right routevalue dictionary?

Oct 28, 2011 at 12:39 PM
bertrandleroy wrote:

Here's what Louis answered:

Yep, thing is routes can be registered from any active shell. Even with one site when you make a config change there's one shell retiring as a new shell is incarnating.

So - for each route to talk to its database through •its• hibernate lense... All complicated by the fact transaction scope is a thread-wide effect... It's just not good. Blows up in your face as soon as looks at you.

•Plus• each call to actionlink, every call that turns route values to a path, •also• rips through the routes match function to find the hit. So database calls in a route would destroy your performance while rendering an a few orders of magnitude more than you'd expect if you're only considering the db calls needed to match the incoming route.

Good news is there's a way to do this we used in the content route before. The route provider for the module gives the routes it makes a reference to a data-bearing singleton. An onchange and inthebackground task feed changing data into the singleton. The route can use the data in the singleton in it's algorithms.

He also implemented a route module that takes care of adhoc routes and the data pump for those adhoc routes. Wouldn't pathauto pump records into that route module, and let that module worry about getting a path to land on the right routevalue dictionary?

Ok, I see how the db calls are a bigger problem than I thought. The thing is, the solution Louis is describing is exactly the status quo that I've been describing as problematic :) The issue with the "data-bearing singleton" is that it reads an entire table from the database during application startup (and iterates over that table, processing it into a dictionary). There's a clear memory and performance question there once your site contains a non-trivial number of pages. As to how many content items would start causing problems I'm not sure, but see for example this thread: http://orchard.codeplex.com/discussions/276222 - we didn't figure out why he was having performance problems after inserting 180,000 routable content items, but I personally I wouldn't rule out the possibility.

So; the Alias implementation can be improved as it is with a better locking strategy, and by allowing individual aliases to be updated in-memory (this is far harder than it sounds, the relevant code is very complex and again has no comments). At that point it's certainly better than current routable, since it reads all the data from a single table, instead of processing content items. But I still maintain it's pretty obvious that having to process an entire database table at startup (for each tenant!) is not a scalable solution.

I was about to say I don't have a good alternative ... but actually an idea just struck me, I have to think on it a bit longer to see if it's viable.

Coordinator
Oct 28, 2011 at 6:15 PM

OK, please keep us posted. This being said, worst case, if the app blows up on hundreds of thousands or even tens of thousands of items, that is not necessarily a death blow to the feature. It just means that it won't be usable on the largest sites, but it will still be tremendously useful on most Orchard sites (which are far from those scales, as even sites that are doing extremely well in terms of traffic often contain only dozens to hundreds of items, at least from my experience). What I'm trying to say is that this may just give us one more reason not to bake too much of this into the core but make it a feature that can be plugged off. A large site can still rely on traditional MVC routes to reach that kind of scale. It works today and will continue to work. There can also be mixed situations where the heavy contents is served through routes, and the rest goes through aliases.

In other words, make it work, and bonus points if you can make it scale to huge sites.

Oct 29, 2011 at 4:53 PM

There is another circumstance that could blow up - someone trying to run 1,000 tenants that each contained only 100 pages would be the equivalent of 10,000 pages. Actually it'd be even worse, because each tenant would be performing a separate database query. That'd be 1,000 database queries on startup - doesn't sound pretty.

Scalability isn't really optional for me. Switching to traditional MVC routes for a large site implies that I'd lose any notion of SEO because I'd have to drop back to /Page/{Id} routes, losing all my nice keyword-targetted urls. So, I feel I have to make this work - hopefully my solution will do this ;)

Coordinator
Oct 29, 2011 at 10:13 PM
Edited Oct 29, 2011 at 10:14 PM

1,000 tenants, again would be a very atypical case. I think with that many tenants, routing would be the least of your problems.

But in both cases, we just don't know. In other words, we are doing premature optimization, and that's never good.

Let's implement as planned and examine later how to make it scale with real data, profiling, etc.

Side note: no, you wouldn't lose your ability to have semantic URLs: what works today would still work. Numeric ids are not the only urls you can handle with routes.

Oct 30, 2011 at 8:02 AM
bertrandleroy wrote:

1,000 tenants, again would be a very atypical case. I think with that many tenants, routing would be the least of your problems.

But in both cases, we just don't know. In other words, we are doing premature optimization, and that's never good.

Let's implement as planned and examine later how to make it scale with real data, profiling, etc.

Side note: no, you wouldn't lose your ability to have semantic URLs: what works today would still work. Numeric ids are not the only urls you can handle with routes.

Surely the current situation is already "premature optimisation" - creating an elaborate in-memory dictionary to store what could easily be held directly in the database? Or did Louis base his statement on actual performance data?

How can routes handle semantic Urls (i.e. "/petes-blog/my-great-blog-post") without either database access or some other way to look up the urls?

Coordinator
Oct 30, 2011 at 8:58 AM

Elaborate? No, the design is made to allow for a specific scenario, not for perf. There is not a bit of premature optimization there.

Routes handle semantic URLs by ending with a wildcard. Then, it's the action's responsibility to do the database access.

Oct 30, 2011 at 9:37 AM
Edited Oct 30, 2011 at 9:38 AM
bertrandleroy wrote:

Elaborate? No, the design is made to allow for a specific scenario, not for perf. There is not a bit of premature optimization there.

Routes handle semantic URLs by ending with a wildcard. Then, it's the action's responsibility to do the database access.

So: why can't we just handle this particular scenario with a wildcard right off the bat, and just do the legwork in the action?

By "elaborate", I mean that the implementation of AliasHolder, AliasStorage, AliasMap, etc. ends up being extremely complicated, and I'm still not sure if I can see the gain.

Note that Louis' comments actually included the word "performance", leading me to believe specifically that there were performance concerns leading to this design decision.

Coordinator
Oct 30, 2011 at 8:49 PM

We can't use a wildcard for that because a wildcard needs to be the last to be considered, after all the other routes had a chance to handle the request (see the priority=10 on routable and 15 on containers -higher value is lower priority (lame, yes)-). Aliases, reversely, need to be first so the feature can take over any URL in the system. The flipside of this is that they need to handle exactly what they need to handle, not more, and let the rest fall through. they are not a wildcard, they are quite the opposite of that in fact. This really is the essence of the feature. So they need to act before all other routes, and they need to handle only what they should. I hope this clarifies.

Louis does indeed talk about perf, but not for situations where we don't have data yet. The situation he's talking about is not a scaling problem, it's a perf issue that you will have right from the start, on any request, on any configuration of Orchard.

Oct 30, 2011 at 9:13 PM
bertrandleroy wrote:

We can't use a wildcard for that because a wildcard needs to be the last to be considered, after all the other routes had a chance to handle the request (see the priority=10 on routable and 15 on containers -higher value is lower priority (lame, yes)-). Aliases, reversely, need to be first so the feature can take over any URL in the system. The flipside of this is that they need to handle exactly what they need to handle, not more, and let the rest fall through. they are not a wildcard, they are quite the opposite of that in fact. This really is the essence of the feature. So they need to act before all other routes, and they need to handle only what they should. I hope this clarifies.

I already understand all of this (otherwise my Plumbing module wouldn't work!) My point was that the routing situation I described ("/petes-blog/my-great-blog-post", or another example: "/vacuum-cleaners/dyson/super-deluxe-2011") has exactly the same constraints - handling it with a wildcard means it would have to be the last thing in the route table, but there might be other wildcard routes - you have a conflict of priorities and somewhere along the line, you need to check a list of Urls to match a constraint. The normal way to do that, at least in MVC that I've seen and worked with, is to look it up in the database.

Coordinator
Oct 30, 2011 at 9:34 PM

...or look up something, yes. Back to square one, then, no? do we need to summarize where we're at maybe? what our alternatives are?

Coordinator
Oct 31, 2011 at 5:21 PM

Excuse me if I missed something, but couldn't we start with an in memory PathConstraint on the route, and if someone needs it, switch to a database driven constraint. It's just a matter of which implementation is resolved, so a matter of enabling the adequate module.

Oct 31, 2011 at 5:25 PM
sebastienros wrote:

Excuse me if I missed something, but couldn't we start with an in memory PathConstraint on the route, and if someone needs it, switch to a database driven constraint. It's just a matter of which implementation is resolved, so a matter of enabling the adequate module.

That was kind of what I was getting at when I mentioned that there wasn't a one-size-fits-all solution.  Basically, depending on your needs, each solution can be a major detriment.  If you have a small number of aliases, you really don't want it to hit the database every time, and an in-memory solution is perfect.  If you have a ton of items, then an in-memory solution would be terrible.  There's a grey area in-between, however I think leaving it up to the end user and making sure that each solution is optimized is the best direction to go.

Oct 31, 2011 at 5:29 PM
Edited Oct 31, 2011 at 5:30 PM
sebastienros wrote:

Excuse me if I missed something, but couldn't we start with an in memory PathConstraint on the route, and if someone needs it, switch to a database driven constraint. It's just a matter of which implementation is resolved, so a matter of enabling the adequate module.

The problem is you can't perform any database access in a Route or a PathConstraint, because a) the WorkContext doesn't exist at that time and b) there is no TransactionScope yet (well, those two things are kind of tied together). The comments Louis made indicate there are technical reasons why things will blow up if you tried to do so (although I'm sure there must be a way the technical limitations could be worked around). But yes, what you describe would be the ideal scenario. The database driven constraint could actually gradually populate the in-memory one, so it avoids any startup delay but is gradually buffered to memory so overall performance ends up the same.

Edit: @ldhertert, you remember when we were hitting all these problems in your module and seeing all kinds of weird behaviour, right?

Nov 14, 2011 at 6:09 PM

Right, been having another look at all this. Some further thoughts/questions:

1. Regarding the object lock; might it be all round better to just use .NET 4.0's ConcurrentDictionary (and related thread-safe collections) instead of using any locking strategy, to ensure thread safety?

2. The new Routable module - is the intention for this to be developed into Autoroute, or should that be a third, separate module? The part it surfaces is called RoutePart2 (presumably to avoid conflict with RoutePart). This might not look so neat in the UI. I'm proposing calling it "SlugPart" instead (and rename the module to Orchard.Slugs). My rationale is that a) we can use TitlePart to provide a title, so literally all this part is then doing is hooking up a slug to an alias; b) this module doesn't really have anything to do with routing, it's just registering a slug with Alias; and c) it draws a clear distinction between this and the old routable (which I can imagine otherwise being a source of confusion)

3. Actually, is there even a need for a new Routable? In part I say this because the module as it currently stands on codeplex is slightly incomplete (e.g. it doesn't even slugify the title), and everything it currently does will be made completely irrelevant by Autoroute. Is it worth the trouble to implement or should the priority just be Autoroute? We could just modify the old routable to use Alias instead of its own constraint, and leave it in place purely for backcompat (although, it does need moving out of core, so the feature can be disabled if not needed)

By the way, I haven't received the contributor agreement mentioned by Bertrand as yet.

Coordinator
Nov 14, 2011 at 7:23 PM

1. That sounds worth trying and profiling.

2. The new routable should be a feature with a dependency on autoroute and in the same module I think. We can do some renaming for sure, but we need to make sure that we have an automatic data migration from existing 1.x instances.

3. We will not do the automatic slugification. I know it's a regression in front-end functionality but unless someone finds a way to make it work, well, we couldn't. The new routable is just surfacing autoroute and alias features. It should have a radio button to switch between auto-generation (in which case the slug is read-only) and manually entered, in which case it's just a text field that gets saved in the alias table. Does this clarify? I don't think you can keep the old routable in place at all and make it work like it used to. the data needs to be migrated.

I sent e-mail to Outercurve for your CLA.

Nov 14, 2011 at 11:51 PM

1. Ok (I'll get everything else working first of course and just bear that in mind for later)

2. Maybe it's better to just call this module Orchard.Autoroute. The question is, what is the purpose of the new RoutePart - is it simply to expose the Autoroute UI? In which case maybe it should just be AutoroutePart. Either way, the first-time migration needs to take all the paths from the old RoutePart and immediately create Aliases for them. This would require a) checking if the table exists (accounting for table prefix) and b) executing insert SQL (accounting for table prefix) or injecting the IAliasService dependency into the Migration. Now I'm not sure if all of that can currently be done from a Migration so some work might be needed there.

3. Presumably Autoroute *will* do slugification, just not the Javascript version - it'll still need to do it eventually when the item is published and the route gets generated. Personally I'm not fussed about the Javascript variant (although, there are ways that it would be possible, the Publish event just needs a property to tell us it's a "fake" publication so we can avoid triggering any events we don't want...)

Coordinator
Nov 15, 2011 at 12:17 AM

2. Agreed. I can see no reason why this couldn't be done in a migration given the proper dependencies. I might be missing something.

3. What I meant is that even the hack that routable currently uses, which is to publish the item in an ajax request and the abort the transaction (a horrible hack that many resent) would not work with Autoroute because some of the tokens in the pattern may require stuff that won't be available from an ajax call. The best we can do is probably only to check for existing aliases when the alias is manually entered (in that case there is no pattern and just one thing to check). That is probably no big deal as in the automatic case there is no user input in any case. I hope this makes sense.

Nov 15, 2011 at 1:12 PM

2. This would mean that Autoroute needs a dependency on old Routable, which sounds like a degenerate situation if we actually want to retire the old Routable. So instead the migration would have to involve SQL to read the old tables, we can't call on IRoutableConstraint if it doesn't exist anymore; but as far as I know there's nothing that surfaces the table prefix for the current tenant, which would be needed to compose the SQL query. At least, there was a thread I saw recently saying the table prefix couldn't be found, I need to check that myself.

3. Yes, that's what I meant by the "Javascript version" (should have clarified this better). Actually, you realise it was me who raised the workitem back in March about that "horrible hack"? (http://orchard.codeplex.com/workitem/17567) - I came up against it trying to write my Content Notifications module, and it's the same problem faced now with Rules. So I'm obviously in favour of doing away with that particular feature!

I realise things get even more complicated with Autoroute because tokens can come from all kinds of different parts. All I was saying was it would conceptually be *possible* with a little creativity to get the Ajax url preview whilst avoiding the Publish hack; but yes it would be extremely difficulty and absolutely not worth it!

Can I just check something from your Autoroute blog post. There's a setting "Enable per-item configuration" - would unchecking this force the author to accept the pattern and not be able to input their own slug, and presumably hide the "Generate from pattern" checkbox?

Coordinator
Nov 15, 2011 at 5:41 PM

2. You can get the table prefix by injecting ShellSettings. But in any case, we are considering building a throw-away "Upgrade to 1.4" module that people would have to install prio to the upgrade. More on that later.

If you uncheck, yes, you abandon control of the url to the autoroute feature and from the content item editor everything is read-only.

Nov 15, 2011 at 6:17 PM

Great - I thought there should be a way. I guess a SQL INSERT followed by a call to Alias would do the trick.

Alias is basically working (backport was easy) I'm just doing some testing and then to work on Autoroute; this shouldn't really take too long, so much of the framework is being provided by existing code.

Nov 15, 2011 at 8:37 PM

Final question: how do we handle the homepage in Autoroute?

I know you said somewhere that the old HomePageProvider would no longer be relevant (a good thing, I agree, the mechanism was a bit arcane!)

So with Alias, obviously we can have a ~/ alias which means we can wire the homepage to any controller/action that we like.

The question, then, is how do we surface this in the UI; in fact, do we even surface it? The user can go ahead and edit aliases manually to change the homepage but this feels a bit clunky. I never hugely liked the "Set as home page" checkbox and you didn't show it in the UI on your blog post.

When "Enable per-item configuration" is checked, the user can set the slug blank to set a homepage; but one thing about this - what do we do about the item that previously was the homepage? We could apply its default pattern at that point and display a notification that this has happened, not sure if that's ideal.

Finally what if "Enable per-item configuration" isn't set - how do we choose a homepage then?

Have you had any thoughts about this?

Coordinator
Nov 15, 2011 at 8:39 PM

Well, having the same mechanism could be nice. You just remove the previous ~/ when creating the new one.

Coordinator
Nov 15, 2011 at 8:44 PM

The home page alias should be no different in principle from managing conflicts for other aliases. If you already have a /foo and attempt to set a second content to that same alias, you will get an error. The question is how do we deal with that error? Is it enough to have a link to the other content in the error message so that you can go and change it?

In practice though, the home page is special. It does hold a special place in people's minds, so we might want to special case it in the alias management somehow, or maybe even give it its own settings screen for discoverability. Any ideas/thoughts?

Nov 16, 2011 at 1:12 AM
Edited Nov 16, 2011 at 1:13 AM

Yes, home page is an odd special case. I just went back and read the information you pasted from Dave and realised it was brought up there. A site setting would be possible, you could have an input for the home page url and have Alias figure it out. Once we have some kind of "Content Picker" interface we could use that, I think that would be the best UI. Beyond that I have no big ideas, it's a tricky one to decide. I'll see how things pan out with the rest of the interface.

I've got Alias working now (my Plumbing module is integrated with it and working nicely) - everything seems fine. I made a couple of small changes (backporting, and a fix for a bit of MVC weirdness). Should I push this up to a fork on orchardalias.codeplex.com?

Nov 22, 2011 at 9:56 PM

@randompete Can you tell us what you changed to make it working? I managed to integrate the OrchardAlias module and I enabled it but I got an error when I try to add a new alias. Thanks for your help.

Nov 23, 2011 at 12:32 AM

@balspeare: Take a clone of my fork at http://orchardalias.codeplex.com/SourceControl/network/Forks/randompete/AutorouteDevelopment, this is the version I have working (although it's still not integrated with Routable).

Nov 23, 2011 at 10:55 PM

Thanks randompete it's working.

May I ask some advice? I tried to use it to create an alias for the base url but it didn't work. I would like to have an alias for example: http://test.mydomain.com which redirects to http://mydomain.com; is it possible to modify this module to do it? or it's too complicated?

Nov 24, 2011 at 12:19 AM

The reason the base url doesn't work is because the HomePage route is still active - you can add a suppression and it works - I didn't push that change yet because it could break existing sites.

Routing to different domains is something I've thought about and discussed elsewhere but right now I'm not sure the best way to do it, could just add an additional column into the Alias table.

Coordinator
Nov 24, 2011 at 12:58 AM

There is the rewrite rule module for external redirects.

Dec 7, 2011 at 8:41 PM

There is now a working demo of Autoroute. The features still need fleshing out but it works for Pages, Blogs/BlogPosts, and most custom content types.

If anyone wants to have a play or help test it, here are the details:

It requires this fork of Orchard 1.x: http://orchard.codeplex.com/SourceControl/network/Forks/randompete/Autoroute

The module itself is a subrepo which comes from here: http://orchardautoroute.codeplex.com/

The project home page has details of some commands you can use to create patterns, since that admin UI isn't there yet.

Known issues: Blogs and Containers still have a certain reliance on RoutePart. Right now there's an issue with viewing blogs due to this, and I still need to figure out how to handle archives because it needs to go thru a separate system. But for most content types it should be fine.

Example routes:

{Content.Slug}
-> /my-page-title

{Content.Container.Slug}/{Content.Date.Format:yyyy}/{Content.Date.Format:MM}/{Content.Date.Format:dd}/{Content.Slug}
-> /my-blog/2010/12/07/my-new-post

blogs/{Content.Author.Content.Slug}/{Content.Slug}
-> /blogs/pete/welcome-to-my-blog

And of course, anything else you want to extend it with using tokens.

Coordinator
Dec 7, 2011 at 10:02 PM

Did you add a handler for ContentItemMetadata ?

Dec 7, 2011 at 10:10 PM

No - it doesn't need one.

The way it works is that other handlers can set the DisplayRouteValues to whatever they like. The AutoroutePartHandler then just uses those route values to create the alias. Alias handles displaying the aliased Url when anything makes a link for those route values. It's actually all incredibly simple, and completely extensible because any module can hijack an item with a custom display controller.

Although, you did just remind me I should probably still be using GetContentItemMetadata to set an Identity with the generated alias.

Dec 7, 2011 at 11:14 PM
Edited Dec 7, 2011 at 11:15 PM

Tricky problem surrounding Blogs: some of the periphery routes and controllers are dependent on the BlogPathConstraint, and it's not immediately clear how to get them onto the new system. The two culprits are Archives and RSD.

1) An easy solution is to make them all use an Id instead of the path. This means surfacing Ids in urls which isn't great (although, there are already Ids being surfaced in RSS feeds).

2) Another option is to continue to maintain the BlogPathConstraint for just this purpose; I don't like this and it's not very neat, and needs extra code to ensure the blogpath and alias stay in sync.

3) Final option (my preferred) is adding some extension points to make it possible to register multiple aliases per content item. The Archive route also needs a wildcard alias to work properly and retain the same Url structure. I think this is possible and not too hard; or alternately I could alter the Url structure a little bit so it's "{blogPath}/Archive?archiveData={archiveData}" instead of the current "{blogPath}/Archive/{*archiveData}" (we really don't want to be registering aliases for every possible combination of archive data parameters!)

4) Ok there's a fourth option - drop the archive functionality entirely from Blogs and use Projector for this instead.

Any thoughts on all this and what best to do?

Dec 19, 2011 at 6:34 PM

Bumping: Have been waiting on a response to the above before I continue ...

Dec 29, 2011 at 1:05 PM

Further Discussion After Steering Committee Meeting

After discussing this issue with the steering group I'd like to go over the solution that was proposed and why I think it might not work; and outline the problem a bit more clearly and what I think might be a solution; and finally to just detail the current status of Autoroute because there's quite a bit more to be done than I'd thought.

Problem Outline

There are two ways to look at this problem. It's either two specific scenarios in Blogs that could be individually tackled in different ways. Or it can be seen as a generic problem that it'd be very useful to have a general solution to.

The generic problem is as follows:

- We have a content item sat on an alias "/my-content-item"

- We then want an action to respond to a Url of "/my-content-item/some/child/path". The action must receive a reference to the content item, but apart from that it could be doing absolutely anything it wants, route-wise.

I mentioned the two specific ways this is required for Orchard Blogs:

1. "{blogPath}/rsd" - this is the handler for remote blog publishing.

2. "{blogPath}/Archive/{*archiveData}" - the handler for archives, and archiveData can take the form {yyyy{/mm{/dd}}} - parsed with a regex rather than routing constructs.

So we could just go ahead and tackle those two scenarios alone. But from my perspective it'd be useful to provide a general system allowing sub-aliases to be easily created. If it's needed twice for blogs then there are surely dozens of examples where this could be useful. An example is "/my-product/{infoTab}" - for displaying product tabs like details, specifications, reviews, etc.

Solution Proposed at Meeting

What was proposed was to use a specialised IRouteConstraint when defining the route for {blogPath}. When I thought about this and looked again at the code, I realised there are a number of problems with this.

1) Alias doesn't have an IRouteConstraint. It doesn't even have a Route. It uses AliasRoute which is a custom implementation of RouteBase. AliasRoute itself performs processing which would then need to be replicated in any IRouteConstraint creating redundancy; and in fact some of the processing would be hard to conceive, since it performs processing in both directions (both in url generation and route interrogation). Perhaps we could change Alias to use a normal route and an IRouteConstraint instead; but this would be a fundamental design change and make the system slightly less optimal overall. Since the current design is one I inherited from the previous developer I assume there's a very good reason for it and I don't want to change things so significantly without serious thought.

2) We don't even know if that route pattern will work. With Autoroute I can set my blog urls to anything I want. For instance; if I simply defined my url pattern as "/blogs/{Content.Slug}" it would break these routes. A route pattern of {blogPath} would no longer match because the blogPath now contains a slash.

There are perhaps other issues with this solution, unfortunately due to the complexity of Alias it's very hard to explain how the overall design works and why traditional routing suddenly doesn't fit very well! If anyone wants to understand the problem better, it's best to look at AliasRoute.cs and Routes.cs from Alias, to see how Alias handles its own routing; and then look at the Routes.cs from Blogs to see the routes that blogs need to implement.

Possible Alternate Solution

I've started implementing something that works and is relatively simple, but still might not be ideal. It's option (3) from my previous post - Autoroute fires an event once the initial path has been tokenised from the pattern. This event gives you a collection into which you can insert additional aliases to any custom actions you like. This means you can construct "/aliased-path/any-custom-path-i-like" and alias it to your own controller/action. I can then implement this event in the Blogs module to create the RSD alias as well as creating aliases for {blogPath}/Archive, {blogPath}/Archive/{yyyy}, {blogPath}/Archive/{yyyy}/{mm}, and {blogPath}/Archive/{yyyy}/{mm}/{dd}. This will perfectly replicate the current behaviour.

Unfortunately this approach creates problems of its own. In some ways it goes directly against the design philosophy of Autoroute, because there's no way to customise these aliases without code overrides. There may be other issues, e.g. when we want to update a batch of aliases because a blog's title has been changed things get more complicated.

What is the Archive route actually doing?

This route is nothing more than a list-by-date mechanism for blogs. Reduced to this requirement, it might be worth considering that there are other ways to implement this behaviour.

Autoroute Status

 

To be done:

  • Unit Tests. No Tests have yet been implemented for any of Alias or Autoroute.
  • UI. Interface needed for managing route patterns. Editor interface needs polishing.
  • Remove Routable. A large number of the ties have been severed, but there are a huge number of places where RoutePart is referenced and some are still conceptually hard (more detail on this in a follow up).
  • Migrations. When removing RoutePart we need to copy all titles to TitlePart, and create default route patterns for core content types. A fresh installation of the autoroute fork will actually create default patterns when the installation recipes are run, but this will need to be performed in migrations as well for upgraders.
  • Permissions. At the demo it was raised that permissions should control whehter a user can select a custom pattern or slug, or whether they are forced to the default.


Dec 29, 2011 at 1:22 PM

Lozalization

As I mentioned above, there are still some ways that RoutePart is fundamentally tied into Orchard in all kinds of places. Actually the only real issue right now appears to be Localization.

This is best explained with this code snippet from AdminController in Orchard.Localization:

            //todo: need a better solution for modifying some parts when translating - or go with a completely different experience
            if (contentItem.Has<RoutePart>()) {
                RoutePart routePart = contentItem.As<RoutePart>();
                routePart.Slug = string.Format("{0}{2}{1}", routePart.Slug, 
siteCultures.Any(s => string.Equals(s, selectedCulture, StringComparison.OrdinalIgnoreCase))
? selectedCulture : "", !string.IsNullOrWhiteSpace(routePart.Slug) ? "-" : ""); routePart.Path = null; }
What's happening there is when you translate an item, it uses a path of /my-item-alias-{lang}

This somewhat disagrees with Autoroute's philosophy of letting the user define their own patterns, and it's generally a problematic way to set the translated path (as the todo already indicates). Any suggestions on this are welcomed. Perhaps we need an event hook so other modules can perform custom processing during localization.

Dec 29, 2011 at 3:35 PM

Another Update

I've pushed a lot of changes. I went with the implementation I described above for rsd/archives - it works and was cheap to implement, and cheap in terms of performance ... although longterm maybe this can still be improved.

I've also pretty much finished the task of eradicating RoutePart and HomePage. There are still some minor issues, and Localization is a problem.

It looks like the biggest remainder of work is tests and migrations - and there is a lot to do there.

Coordinator
Feb 4, 2012 at 7:56 PM
Edited Feb 4, 2012 at 8:33 PM

Quick status on the autoroute side. 

I am currently merging all the work made by Pete into the main Orchard repository. There will be two development branches for a few days, autoroute and 1.x, until we decide to merge then. Any change not related to autoroute will be done into 1.x and merged to autoroute. This will give some flexibility to fix bugs that early adopters want fixed in 1.x. Though you are encouraged to use the autoroute branch if you want to try your modules with it, or if you just want to use those great new features.

Also, if you want to migrate a 1.3, or 1.x website to autoroute, then:

- run autoroute branch on your website
- go to /admin and authenticate, even if the homepage doesn't work 
- enable UpgradeTo14 module
- Click on Migrate Route in the admin menu
- migrate each content type which is listed 

Please report any issue you might find or any incompatibility with existing module

Mar 18, 2015 at 1:17 AM
I have worked and finished adding Autoroute Pattern Localizations on a local private branch. So far, it is working against a new installation of Orchard and also with already made website using 1.x branch, working with Pages and also custom content types. Would it be something that you guys would like to see me demo or just share on a github public repository, pull request ?
Developer
Mar 18, 2015 at 11:22 AM
A demo would be great.