Orchard 1.7 Media Management feature team

Topics: Announcements
Coordinator
Nov 6, 2012 at 8:56 PM

This thread is to discuss media management improvements in Orchard 1.7

Developer
Nov 7, 2012 at 11:40 AM

"Ideas for a new Orchard media management" has a bunch of ideas discussed.

Nov 7, 2012 at 1:52 PM
  • Add more TinyMCE plugins by default
  • Add a separate provider to store non-public files (look at it as a 'data storage')
    • We have a custom AppDataStorageProvider, guess what it does ;) - we use it to store files that should never be directly accessible (we use it for uploads, ..)
    • You could move the RecipeJournal folder to such a provider
  • Extension points for managing different kinds of media (flash, pdf, avi, images, ...)
  • Optional ability to 'strong-link' to media (if you move such a media to another location throuh the interface, the rendered links are updated too)
Developer
Nov 7, 2012 at 1:57 PM

"Add a separate provider to store non-public files (look at it as a 'data storage')"

See "Hidden media folder (like RecipeJournal) for modules to store media files that shouldn’t be directly accessible from the file browser (e.g. generated thumbnail images)?" in the linked discussion.

Nov 8, 2012 at 1:36 PM

Drag and drop media would be very nice. Seen it in other systems (CMS) also and is a very convenient way of uploading for the users.

Coordinator
Nov 8, 2012 at 11:01 PM

@piedone: I was thinking that we could provide a way to limit access to specific folders actually from the admin side. This would work too. I have talked with a customer which would like to restrict a user to a specific folder, this is also something related.

@Znowman: Definitely. We should be compatible with new HTML5 features, like drad and drop (File API). I think we should also be able to use this new metadata standard for media resources in web pages which lets you know what is the url of the media that a page displays. Someone could remind me the name ? The idea is that you paste the url of the youtube page, and it will extract the url of the media directly from it.

1- Manage different types of medias
2- Add metadata on medias. Custom fields ?

Coordinator
Nov 9, 2012 at 1:14 AM

+1 on restricting a user to a folder. This is super-useful for user-provided contents, such as forums.

Developer
Nov 10, 2012 at 1:03 PM

How can a user's media access made restricted? I mean, restricting it on the admin UI is fine, but I guess it's not simple for the static files?

Coordinator
Nov 11, 2012 at 6:10 AM

You won't be able to block access to individual files (they are all public after all) but that doesn't mean you should list them. What a user sees can be restricted to his own folder.

Developer
Nov 11, 2012 at 10:44 AM

Sounds like a bit of a security hole there. Are you sure there is not way to block access to files using a filter of some sort?

Coordinator
Nov 11, 2012 at 6:09 PM

This is not a security hole. You just want some users to be able to save files in specific folders. You also want some user to pick images and add them to content from specific locations. It's not about restricting the access to these files, just preventing how they are used.

Developer
Nov 11, 2012 at 6:11 PM

Disagree. It should be about restricting users to a set of folders. If you make it so that some users cant see stuff, but if they guess the filename and location, then they can see that file... seems wrong to me.

Coordinator
Nov 11, 2012 at 8:00 PM

Everyone will have read access to everything in media. This is not going to change. If you need read-restricted media, build your own feature, as this is not it. This is about restricting who can write where in the media folder. Hiding the other folders from the admin or front-end picker ui is not a security feature but a convenience: why show stuff that you don't care about?

Nov 11, 2012 at 10:03 PM

One small question, are the media inclusion in Orchard  working transparently from https pages (ie, without the warning about non secure......)

Coordinator
Nov 12, 2012 at 1:19 AM

Yes. All media should be included by relative paths so they should work fine with SSL. They do work on my own site.

Developer
Nov 12, 2012 at 7:18 PM

Why cant restricted media be part of this feature?

Developer
Nov 13, 2012 at 10:37 PM
Edited Nov 13, 2012 at 10:37 PM

+1 for restricted media as Nick mentioned. It's a demanded and useful feature in lots of possible scenarios. And it seems logical - if we would treat files as content items we should be able to apply View (which means download) permissions to those as well.

My loose thoughts:

  • Return FileStreamResult instead of ShapeResult from Contents/ItemController for items of type/stereotype "Media" (this way we could eg. utilize AutoroutePart for permalinks ootb). Performance drawback of returning file through controller is actually very small, especially with output caching in place.
  • Separate download and preview of a given media item, as a type setting ('Enable preview'). Useful for embedded media like videos, music, presentations etc.
    • If preview is enabled, item display URL would render a display shape, as for any other content item (with tags, title and possibly other things like embedded video player and/or download link). Otherwise it should return file contents as described above.
Coordinator
Nov 13, 2012 at 10:41 PM

@Piotr I think we should separate the content item from the associated media mostly for performance reasons. You don't want a simple file to be served by the Orchard pipeline including routing and events. That would slow all websites, and also remember the session locking stuff ...

Developer
Nov 14, 2012 at 12:02 AM
Edited Nov 14, 2012 at 12:41 AM

@sebastien Yes, performance is my only concern. There are couple of things to take into account, though:

  • By turning the session off on item "Download" routes we can get async requests, which makes things considerably faster (so eg. an image would load in parallel with the rest of the page, not after).
  • We can add caching settings for media item (using output caching and Cache-Control header). Frequently accessed items would benefit from that.
  • Events (if we're speaking about Rules engine), like eg. "File downloaded" may actually be pretty useful!
  • Buffering large files

I've used this approach a couple of times in Orchard (even with the image data fetched from external service) and the actual performance impact is negligible, if session is turned off and caching is in place. Shape rendering is still the most time-consuming task that impacts page load time - if it doesn't happen (like when returning an image, but also in my experience with SignalR) Orchard is getting really fast and response times go down by an order(s) of magnitude (depending on how complex the site shape structure is). That being said - I wouldn't really bother about the performance impact if the tools to minimize it are in place, unless someone comes with an insane idea of putting thousands of media-based images on a single page...:)

Coordinator
Nov 14, 2012 at 12:37 AM

Even if it is possible to optimize that code path, it is still going to be considerably slower than IIS serving it directly (and no, caching is not necessarily the answer here). We have hard evidence of that. Nobody is saying that it wouldn't work, just that we don't want it in the media feature. It is fine as a separate module.

No, the performance impact is *not* negligible because we cannot always turn session off and caching on.

What you are describing is a separate feature. Feel free to build it.

Developer
Nov 14, 2012 at 1:12 AM
Edited Nov 14, 2012 at 1:21 AM

@bertrand I agree that it's still going to be a bit slower, but that also depends on a scenario. If we utilize the cache-control header there may be no requests at all in a given interval, making the performance impact really low.

I don't understand why we cannot always turn session off? I meant that session may be turned off for a particular route (the download route) so it will run in parallel with other resource requests. Even if session is turned globally on (which in 99% of scenarios is true), that one may have it turned off (using RouteDescriptor.SessionState).

Caching setting would need to be set either per-type or on particular item level, but there is always some value appropriate for any item, unless you'd want to output a dynamically-generated image that changes over time which is not the case here.

Btw - apparently I may need to build that feature by the end of this year this way or another, but it seems logical to include it within Orchard core, as otherwise this will a bit misleading. If we have content permissions and media files are going to be content items, people will start using those together and to great surprise notice that files are actually not being secured at all...

Coordinator
Nov 14, 2012 at 1:32 AM
Edited Nov 14, 2012 at 1:32 AM

No, not "a bit", a lot. Seriously. You cannot always use caching, and caching doesn't work on the first hit (duh). Please, this is a separate feature. Really. I veto it.

The additional permissions you will have are writing permissions and will be clearly labelled as such.

Developer
Nov 14, 2012 at 2:02 AM
Edited Nov 14, 2012 at 2:12 AM

All right:) But to be clear - I have a feeling that we're talking about two different things and scenarios.

Maybe we need a distinction between "files/images" (Bertrand I guess it's what you're mainly referring to) and "media content" (it's what I have in mind)?

  • Media content = files wrapped up in a content item. Subject to security checks etc. Heavier, but allowing much greater flexibility. This would be suitable for eg. videos, music, invoices, reports, some user-submitted files, attachments, digitally-delivered content and everything else that you need to have full control over and the additional milliseconds per download are not an issue.
  • Files/images = items one inserts using media picker, so exactly what we have now. Light, directly accessible by everyone. Those may be later wrapped up in a content item if needed (or directly from media picker, by checking some checkbox).

Or

  • Make everything a content item and let user choose file accessibility. Whether a file needs to be accessible directly or not could be then specified on item level (checkbox). If not, the underlying file is moved to some publicly inaccessible storage space and accessed via controller action instead.

Or

  • Make everything a content item and publicly accessible. But use item identity (Guid) as a filename instead. If user is not allowed to view the item, simply do not display the link. Smells a bit like "security through obscurity". Should work unless user knows the exact file Guid.

One more thing about the requests and performance. There was also an idea (by AimOrchard) of having strong-links (permalinks) for media. That would require a request to application (which would serve it or redirect to a static file) if we decide to have it, so it's not about securing items only.

Coordinator
Nov 14, 2012 at 2:12 AM

Yes, we need a distinction, which we'll have easily if you implement what you call media content as a separate module. What I've been saying all along ;) Options 2 and 3 are out of the question (for the core feature).

Developer
Nov 14, 2012 at 3:13 AM
Edited Nov 14, 2012 at 3:22 AM

I'm not convinced, but well... you're the dictator here;)

I can build that as a separate feature, sure, but we'll have to agree on the architecture and possibly some extension points first. There are a couple architectural questions that come to my mind right now:

  • If we're going to have a "Media" stereotype, I guess we'd need a MediaPart (similar to widgets) with data (what data?) required by the new Media Picker?
  • How is the new Media Picker going to handle non-image files?
    • Maybe we need to introduce an additional shape display type that would be used render item's picker screen, instead of the current hardcoded one that has width, height, alignment etc.? Eg. "PickerSummary" or "MediaSummary".
    • How can the picked item define a tag in which it has to be rendered in (like eg. anchor, video) instead of the default img?
  • Do we need to extend content metadata to support additional links? Currently we have DisplayRouteValues to display an item, but maybe it would be ok to also have ThumbnailRouteValues and PreviewRouteValues (with fallback to DisplayRouteValues), plus corresponding extension methods on HtmlHelper and UrlHelper to render appropriate links.

I was also thinking about introducing an IMediaStorageProvider for being able to alter media storage logic on item level. Multiple implementations of that interface could coexist and it could be specified in type settings which one to use for that type for both storage an building public URLs. The default implementation could fall back to default storage mechanisms (FileSystemStorageProvider and AzureBlobStorageProvider). This would be useful in scenarios when particular items need to be stored in database/cloud/CDN/via FTP/somewhere else and accessible from that location by some custom URL. 

Coordinator
Nov 14, 2012 at 4:38 AM

The new media picker can handle non-image files the same way the old one does it: it just doesn't attempt to display a thumbnail. We can refine on that by adding a concept of media type handler that can take care of the thumbnail and of extracting any metadata that needs to be extracted. It could also handle the summary, yes, and how the media renders on the front end, through overridable templates. Looks like drivers and shapes, doesn't it?

I think the solution to generate URLs for thumbnails and previews that is implemented in the media processing module is more elegant.

The storage providers already exist, but it may be a good idea to adapt the system so that more than one can be active at any time. That could enable your extended media module to live side-by-side, seamlessly with the core media module.

Mar 22, 2013 at 2:45 PM
I've watched the latest Orchard CMS Meeting. For @sebastienros : Just want to reiterate what I've already said before in the 1.7 inline editing feature subject. You guys should think about using some more feature complete Javascript library. Knockout.js is good but misses some feature that backbone.js have.

Routing !
Using IFrame is considered back pratice. You guys should use AJAX to reload content and some routing (history.js like).
Backbone.js is having a history.js routing like feature ; not Knockout.js

ORM
Knockout.js is not an ORM vs backbone.js is.

Knockback.js is bringing the 2 librairies together.

In the end the idea is to offer something different than what has already been made before.
IFrame = bad
AJAX = good

Using a history.js like feature in Orchard for navigation should be also considered for those who would like single page navigation websites.
The Orchard admin could be faster if using history.js navigation other than making POST and GET on every pages. Drupal 7 has it but using JQuery BBQ and mixing it up with history.js makes the admin overlay feature break. So, at some point, I think you guys need to make decisions for the core of Orchard and just like MVC is a dependency, those librairies should be too.

You guys need to make a choice for all of us that don't know a thing about JQuery or AJAX. You need to commit over a library someday, else it will always look like a traditional POST/GET form CMS.

My humble opinion
Coordinator
Mar 22, 2013 at 5:40 PM
I don't want to force plugin developers to use a specific library for import modules. Thus we need to use an IFrame to allow postback scenarios. Currently there is almost no AJAX at all in the backend and it has been a thoroughly thought design decision. I think the Media Manager is the first to include such features.

Knockout is a case by case decision right now, because we don't want to settle on a specific library all through the codebase. We don't have time for it right now. This is something that we could think about for a v2 for instance, and be assured I will dig into backbone.js to see how it can help us.
Coordinator
Mar 23, 2013 at 7:57 AM
I see a lot of confusion here. A CMS manages content sites, not SPAs. Content sites have traditional navigation, and too much client-side logic is actually hurtful (for SEO for example). Even the admin has no business becoming a SPA. It doesn't need client-side routing, it has a perfectly fine mechanism for history (the browser's history feature) and it doesn't need the complexity of such frameworks. It works just fine and is easier to manage and extend the way it is.
A "traditional POST/GET form CMS" is what it is, get used to it. There is no good reason why it should be different. SPAs are trendy, but that's not a good reason to make everything a SPA.
As Sébastien said, the media picker is the first part of the admin that justifies including a binding framework. Knockout happens to be a great fit for that feature, but that's not an endorsement or a general recommendation.
Mar 23, 2013 at 4:45 PM
Edited Mar 23, 2013 at 5:18 PM
I was just making an example with Drupal that is actually using SPA in his admin. I agree with you that this is a trendy feature.
The admin should work with "traditional POST/GET form" and there is no problem with this design.
The only thing I say is that if you guys want to eventually innovate you will need to commit on a library or another.
And when it will happen you will eventually need to support both type of history navigation since some people will still prefer the old fashion way.
I speak for the admin side of Orchard.

Using a binding framework or any other library in Orchard does'nt make it a requirement for those who write modules. But using a ORM like backbone.js would reduce redundancy of client-side scripting if used the right way. Again, you can't force people to use it even if it becomes included in Orchard client-side script libraries. Just like some people will prefer using prototype.js rather than JQuery ...

Client-side script libraries becomes a requirement only when included in Orchard Admin. If you start using Knockout.js then it becomes a requirement for Orchard to work. And that's where it becomes paradoxal when you say it's not an endorsement or a general recommendation. You are using it and therefore I will ...

Then it kind of answer the question about if there should be a logic abstraction to separate these 2 kind of design paterns somewhere in Orchard. Module with dependencies and/or separated project.

Advanced UI for Orchard Admin.
Coordinator
Mar 24, 2013 at 6:58 AM
We're using Tagit, and a few other libraries here and there, but that doesn't mean we're recommending everybody does the same. In the case of nHibernate, we had no choice but to be prescriptive, because it's so pervasive. No such constraint here. There is no good reason to prescribe a client-side framework at this time. As a matter of facts this is getting a little off-topic.
Mar 27, 2013 at 12:46 AM
Is the media management module in the Orchard mercurial repo yet?
Coordinator
Mar 27, 2013 at 1:46 AM
No, I am keeping local for the moment. I will push it when it actually works, which should be soonish.
Mar 27, 2013 at 3:34 AM
Cool, no hurry. The demo for this today was very cool, BTW.

I was asking because I wanted to see how you handled all the knockout integration with Orchard stuff.
Developer
Mar 27, 2013 at 11:07 PM
sebastienros wrote:
No, I am keeping local for the moment. I will push it when it actually works, which should be soonish.
I have to remember that word when clients ask for ETAs :)
Apr 5, 2013 at 4:08 AM
Edited Apr 5, 2013 at 4:09 AM
I like what I've seen from the latish demo ;)
Apr 10, 2013 at 2:19 PM
A very important feature that can make life easier, dynamic regeneration picture size. It is also associated with the Orchard 1.7 default theme feature team
Mainly in the Media directory is loaded large picture and the small is created by dynamic recovery. It would be great to see this feature in Orchard 1,7

Thanks
Apr 11, 2013 at 2:19 AM
Another suggestion for videos and audio files would be to support to add links from streaming url like RTSP, RTMP, Smooth Streaming, m3u8 (for ipad/iphones).
Coordinator
Apr 11, 2013 at 3:43 PM
Dynamic images are already implemented for 1.7. Part of the Media Processing module.
Apr 11, 2013 at 6:17 PM
The new module looks very nice in last demo. Great job. If Orchard could make some kind of DLNA and open some streams to extrenal device by itslef it could be killer...
Apr 12, 2013 at 5:39 AM
sebastienros wrote:
Dynamic images are already implemented for 1.7. Part of the Media Processing module.
Tell me, please, where can I download this module?
Apr 12, 2013 at 8:26 PM
CSADNT wrote:
The new module looks very nice in last demo. Great job. If Orchard could make some kind of DLNA and open some streams to extrenal device by itslef it could be killer...
I found this on codeplex while searching on Google :
https://net7mma.codeplex.com/releases/view/99094

Anyway I think a streaming server should be a separate software. Orchard just need to link and playback all these media types.
Coordinator
Apr 12, 2013 at 9:43 PM
Edited Apr 12, 2013 at 9:44 PM
magisterium: It's part of the bundle I have to publish ... sorry
And it will require 1.7 because of inherent changes in the StorageProvider implementations
Apr 13, 2013 at 6:52 PM
Multiple File Upload
Coordinator
Apr 18, 2013 at 7:04 PM
The new Media Library module is available on the 1.x branch.

Enable it, and you can also enable the Image Editor module. There are lots of known issues and missing feature but feel free to open bug in order to track the progress or just ensure that's it's a known issue.

You can also suggest features or improvements, which we will triage.

Thanks
Apr 18, 2013 at 7:42 PM
Cool! Gonna try it out :)
Apr 18, 2013 at 7:47 PM
Going too fast, I am just on Workflow....:)
Developer
Apr 18, 2013 at 7:49 PM
Yaaaaaaaaaaaaay !! :D
Apr 18, 2013 at 7:57 PM
Image

Thanx Sebastien ! :)
Apr 21, 2013 at 5:30 PM
Apr 22, 2013 at 12:17 AM
I'm getting empty images even if the images are uploaded. I'm using SQL CE fresh intall.
Coordinator
Apr 22, 2013 at 6:32 PM
Now the module is publicly available, I might want to use you.

There are some extensibility points that need to be filled with concrete implementations. The most obvious one is the Media Providers list. Right now we can import media items from the local computer, or using a web search (BTW, I need to make it configurable with the API key).

The idea behind this feature is to let devs add new providers through modules if they are not included by default. It includes all those ones:
  1. Media Storage
    • The ability to browse the current storage folder (/Media) and import those files as Media Items. This is a mandatory one as this will allow existing installs to import their current media files to media items. It should be included by default, and reuse some UI code from the Media module. The media items will point to the existing files.
  2. Dropbox
    • Opens an OAuth dialog to connect to Dropbox and let the user pick some media items from there. By default it should clone the image locally like the Web Search does. Optionally it could create media items pointing to the public dropbox url, acting as a CDN.
  3. Skydrive
  4. Flickr
  5. Google drive
  6. Camera
    • Show the local camera, and let the user create a snapshot. Doesn't work on every browser but nice to have for demos.
  7. Url
    • Let a user enter or drag 'n drop the url of a media, or a page containing a media. If a page, then use the OEmbed protocol to discover the meta data and the location of the media. Optionally let the user decide how to render the media locally (OEmbed provides this information).
  8. Facebook, Instagram, Github, ...
    • Not really a priority but could give some ideas
You might have some other ideas so please share them, fell confident I can hate them ;) But I might also love them !
If you want to start on some of those ideas, please make a public announcement, and we can talk about design ideas, and how it has to be done in order to make it in core.
Coordinator
Apr 22, 2013 at 6:33 PM
I think Url, Flickr, Dropbox/Skydrive should be implemented in priority order.
Apr 22, 2013 at 7:12 PM
OAuth is getting more and more useful ....
Apr 23, 2013 at 2:55 AM
I installed latest and it works except for videos. I'm going to try to find some time to work on it. Though I can't make promises since I have a lot of work already.
Apr 29, 2013 at 11:52 AM
Is there any plan to have the new Media Library Picker replacing the ImagePicker in the editor tinymece and tinymce deluxe ?
Coordinator
Apr 29, 2013 at 6:52 PM
Yes.
May 3, 2013 at 8:55 PM
Hi. I just looked at the Media Library plugin and it looks great. Congrats. One question:

I was able to drag the images but when I drop it, it returns to the original position. I mean, is there any intention to allow sorting in some way inside media library?

Regards,
Gabriel
Coordinator
May 6, 2013 at 6:41 PM
The drag and drop is to move media items across folders, not to arrange them. There is no notion of order, unless you are using a Media Library Fields and then you can order your selection.
May 15, 2013 at 10:08 PM
What I've done is added to my ProductContentType a Media Library Picker Field. I then edit the product, and select a bunch of images (but would be nice also to simply type in a tag and have the picking done for you).

But my main question is now, how can I render the images and or have a Image Gallery Control present the images on the front end? I have 2 types of Media Library Picker Fields, one are documents and another are images and being able to present these would be nice.
Coordinator
May 15, 2013 at 10:54 PM
Edited May 15, 2013 at 10:55 PM
Like any other field requiring customization when rendered, you need to either alter the template used for the whole content type, for all the field instances, or for just this field instance.

To do it at the field instance level, just copy the file Modules\Orchard.MediaLibrary\Views\Fields\MediaLibraryPicker.cshtml to your Theme's View folder.
Rename it to Fields.MediaLibrayPicker-CONTENTYPE-FIELDNAME.cshtml

You can also use Shape Tracing to navigate through the available properties in the model.
May 16, 2013 at 12:24 AM
Yes yes. I have found the shapes. I'll share my customizations for others when im done but it would be interesting to see others as they adopt this. Thanks for the info to support the direction im going. Nice work!!

Sent from my Windows Phone

From: sebastienros
Sent: 15/05/2013 5:55 PM
To: [email removed]
Subject: Re: Orchard 1.7 Media Management feature team [orchard:402143]

From: sebastienros

Like any other field requiring customization when rendered, you need to either alter the template used for the whole content type, for all the field instances, or for just this field instance.

To do it at the field instance level, just copy the file Modules\Orchard.MediaLibrary\Views\Fields\MediaLibraryPicker.cshtml to your Theme's View folder.
Rename it to Field.MediaLibrayPicker-CONTENTYPE-FIELDNAME.cshtml

You can also use Shape Tracing to navigate through the available properties in the model.
May 17, 2013 at 1:29 AM
Edited May 17, 2013 at 2:34 AM
Can we get two features in the gallery selection screen (the one with the thumbnails and the check marks when you click on them.
  1. Select all images in a gallery (I want to select all images in a folder)
  2. Sorting of the files. Right now all images appear not to be in any order. Files are named in a pattern but broken up without order.
And for some reason if I put the Media Library Picker Field as a field in a content-type, then make that a widget, the library only display 10 results. I'd like it to display as many as I select.
Sep 20, 2013 at 1:50 PM
Orchard 1.7.1 Media Manager. I am keen to implement this myself sooner rather than later, but thought worth mentioning that it would be great to have a setting/option to display:

1) Just thumbnails
2) Just filenames (and to defer thumbnail creation until needed)
3) thumbnails with filename (e.g. similar to how the mediapickerfield displays the thumbnails

When uploading 100's of similar looking images a filename is essential to be displayed and the thumb pretty useless. Unless I have overlooked something?

In addition, AzureBlobStorage can do with being able to customize the container name from "media" to whatever. Currently I have hacked this into ShellSettings, similar to the table prefix. This way one storage account can serve multiple sites without going down the tenancy route.
Sep 23, 2013 at 6:09 PM
Just to keep track and share my thoughts: (likely in the wrong place, but heck to that)

Previously used the MediaFolder module to allow a shared file server to store media files. During the process of moving this to AzureBlobStorage (using the core Azure module in 1.7) have come across a few hurdles, primarily that SSL = buggered when using (or rather, wanting to use) a custom domain on the storage account and the lack of SSL support thereof. (Note that the MediaFolder module I believe is missing a few simple implementations of IStorageprovider new since 1.7)

Taking the same approach used to previously adapt the MediaFolder module, I am now considering:

1) As mentioned in the prev post, kill the "media" container and replace with containername = tenantname or similar (I have added a dedicated ShellSetting)
2) Optionally drop the evil "root" from the url, e.g. "/Default/" which is only really needed when using the default filesystem provider
3a) Allow the AzureFileSystem to return a full path to the container/blob, but also a relative "/Media/blob" path, e.g. when the request is using SSL, or:
3b) AzureFileSystem always returns a relative path and delegate the function of determining the full path (site relative, azure storage, cdn, ssl, no ssl, etc.) to a third party, most likely a filter mechanism.
5) Update MediaPicker mechanism to only insert relative /Media/ paths anywhere where an img tag is inserted, my guess is changing AzureFileSystem will do the trick automatically.
6) Add a controller to deal with /Media/ url's to serve up the media file as if stored locally (under the site's domain), solving the SSL issue. Re-use the clientcache fork created on the MediaFolder module to ensure client-side caching.
  • We could also use the IHtmlFilter to rewrite media links to appropriate paths (as previously achieved with the LinkRewrite module, but this module suffered from the lack of support for the IHtmlFilter in various custom modules).
  • A filter mechanism is important to make the decision "to leave relative or to rewrite absolute". Not only the request being SSL can be important to use relative, but also e.g. the user agent (search engines, I say no more), the client location, etc.
What I am hoping to achieve, is that all media url's, at least internally to Orchard, are relative to e.g. a "/Media/" path. How the url is rendered to the client is a "last-minute", configurable and dynamic decision (and decoupled from the storage mechanism?). Ideally I should be able to swap one storage provider for another with relative (pardon the pun) ease.

Please note that the above refers to a webfarm scenario consisting of multiple, load-balanced / redundant VM's where previously media was shared on a file server. Using AzureStorage negates the need for a VM running as a file server, wasting precious CPU cycles on storing files.
Coordinator
Sep 23, 2013 at 9:37 PM
Wouldn't you be adding a level of indirection in front of every single media file?
Sep 26, 2013 at 3:20 PM
Precisely that, but isn't this already more or less the case anywhere a Media field is used?

In the meantime, I have been refactoring the AzureFileSystem to accommodate this. The result is that the AzureFileSystem (and the default FileSystem) are only concerned with the "how" but not the "where", this is left to a chainable "IStoragePathFilter", allowing any module to modify url's as they see fit.

The out-of-the-box config can be changed from the Admin settings, with separate settings for ssl and non-ssl requests, dropping the tenant name, virtual relative /Media/ folder, a custom endpoint, etc.

If a container name is appended to the endpoint in Web.Config, this container will be used instead of the default "media". I am hoping to also move the storage connection string to the Admin settings - by default the new AzureBlobStorageSettingsPart could pick up the connection string from Web.Config for backward compatibility, but can then be overridden (or set in the first place) from the Admin.

Listing the objectives covered so far:

1a) Done, The container can be specified in the BlobEndpoint in the connection string or will use the default "media" container if omitted.
1b) Done, An option has been added in Admin to include or exclude tenant name in public urls. (the tenant name will never be stored in the mediapart)
2) Done, See 1b, "Default" is just another tenant name
3a+b) Done, as specified in IStoragePathFilter and implemented in two default filters: FileSystemStoragePathFilter and AzureBlobStoragePathFilter
5) In-Progress, probably no changes to MediaPicker, MediaService etc. required
6) In-Progress, rudimentary MediaController done, needs addition of client cache headers and relevant response codes, e.g. 404's (addition of ETag to IStorageFile?)

To do:
7) Some short duration caching could be be beneficial for the public URL's and a signalled cache for settings a must.
8) I will write a "IfUserAgentIsGoogleThenUseSiteRelativePathInsteadOfCDNStoragePathFilter", but that will be a bit specific to my needs. But anyone can ping me for the source, of course.

A few more days left I reckon. I will commit what is ready to the azurerelativestorageprovider fork later today.
Coordinator
Sep 26, 2013 at 5:44 PM
No, there isn't indirection in front of the file itself, only in front of its url in the page it's embedded in. This can't be the default. Cool stuff otherwise.
Jul 18 at 4:39 PM
bertrandleroy wrote:
The storage providers already exist, but it may be a good idea to adapt the system so that more than one can be active at any time. That could enable your extended media module to live side-by-side, seamlessly with the core media module.
I am at the point where I would like to utilize two storage providers (and keep a seamless system for my clients). Has this been done by anyone yet? Here is my goal:
  • I have client's with the need to have some files in a "Secure" folder (right now this is accomplished with web.config)
  • I want to move to cloud storage (in this case Azure Blob) where web.config is not supported to make a "folder" secure
  • This being the case, I want to keep the native storage provider (i.e. AzureBlobStorageProvider) in effect so non-secure files are resolved directly
  • I will set up a new blob storage that is private and would provide my own storage provider for that one so files are resolved through my control logic
As of right now, I cannot find any extension points in the Media Library allowing me to support two storage providers. One solution proposed above of having the storage provider controlled by the content type could work also (might make more sense that having it be a "folder"). The media stored in the content type could be saved/retrieved from a separate blob / storage service.

Right now the Media Library seems very extensible when it comes to working with media in one storage system, but does not appear to accommodate multiple storage systems. Am I wrong on this? Please let me know (and glad to program a solution if one appears available).
Developer
Jul 18 at 4:50 PM
You could create an IStorageProvider implementation that uses some logic to determine which storage mechanism to use, then you could make use of the existing ones inside that (not sure if there is a better way than instantiating these "inner" storage providers by hand).
Jul 18 at 5:26 PM
Edited Jul 18 at 5:31 PM
See, that is where my head went but I cannot get over this one mental hurdle. After I suppress dependency (to make my Storage Provider the "Master") how can I go back and call the other storage providers (that I had just suppressed)? Perhaps my understanding of suppress is incorrect and I can just call them directly (rather through dependency injection). I will give that a go and hope for the best.

To be honest though, I would rather implement a "permanent / generic solution" Perhaps this process will reveal one. Ideally the storage provider could be set on the MediaPart and trickle back to the storage provider to "hint" at what storage provider is desired to be used. I might actually set my solution up to hint at that. Here is what I am thinking:
  • Storage Provider Hint Settings
    • Provides settings to indicate the default storage provider
  • Storage Provider Hint Part
    • An attachable part that contains a text field that you can type in the storage provider name / something to indicate desired provider
  • Storage Provider "Master" implementation of IStorageProvider which (somehow) obtains access to the content item and checks to see if it contains the StorageProviderHintPart, if it does, it uses that to determine the storage provider, if it doesn't, it falls back to the StorageProviderHintSettings
That of course is no way perfect because what would happen if someone changed the default after some media was created. It might be the start of something useful though, glad to entertain your thoughts and I will try to get this up on it's own repository.
Developer
Jul 18 at 5:32 PM
As mentioned if you want to create a storage provider that uses multiple other storage providers internally I don't know about a better solution than to instantiate those others by hand (that is, unless there will be some more specific interfaces for them e.g. ILocalStorageProvider).