A summary of date/time localization and calendar improvements in 1.9

Topics: Announcements, Localization, Writing modules, Writing themes
Developer
Aug 7 at 2:20 AM
Edited Aug 7 at 5:48 PM
I am about to merge the feature/datelocalization branch back into 1.x for inclusion in the upcoming 1.9 release of Orchard. The following is a summary of the fixes and changes, in case anyone wants to know.

My goal has been to fix all known issues related to date/time localization in general, and custom calendar support in particular. As far as I know, I have reached that goal.

FUNCTIONAL FIXES/CHANGES

Parsing non-Gregorian dates (e.g. Persian calendar dates) now works as expected in all cases. Before, if the user selected or entered a date which was not ALSO a valid Gregorian date, the parsing would fail. For example, the date 1392-02-31 is a valid Persian date but not a valid Gregorian date; selecting or entering that date caused the parsing to fail.

Fixed issues:
DateTime fields configured in date-only mode are no longer corrupted by time zone conversion. Before, when using a time zone with a positive UTC offset, the time component would be assumed to be 00:00:00 and the combined date would be converted to UTC, resulting in the date component being changed to minus one day. DateTime fields now bypass time zone conversion if configured in date-only mode, and bypass calendar conversion if configured in time-only mode.

Fixed issues:
The correct localized month names are now printed when using the Persian calendar in combination with the fa-IR culture. Before, the localized Gregorian month names were printed even though the Persian calendar was used. Additionally, the names and formats for all other cultures are now calendar-specific, whenever such localizations are provided by the .NET Framework.

Fixed issues:
Date/time tokens are now properly calendar converted. Before they were only formatted using culture-specific formats, but no calendar conversion was performed.

Additional date/time tokens have been added. The full set of tokens is now: Date, Since, Local, Short, ShortDate, ShortTime, Long, LongDate, LongTime, Format.

Genitive month names are now used where appropriate for cultures where it is applicable. For example in Catalan, when printing a date using a format containing a day component, the genitive month name "de juliol" is used, but when printing a date using a format without a day component, the non-genitive month name "juliol" is used. Before, non-genitive month names were always used.

BREAKING CHANGE: The date/time tokens "Date.Format:<formatString>" and "Date.Local.Format:<formatString>" now only support custom date/time format strings. Before they also supported standard date/time format strings (but it was hardcoded to the culture and did not respect localization, so it really only sort-of worked). The migration path is to use one of the new tokens where appropriate, or a custom format strings where necessary.

CODE CHANGES/REFACTORINGS

IDateServices has been renamed IDateLocalizationServices and extended to allow more control over the conversion process using an optional instance of the new DateLocalizationOptions class.

A new service abstraction IDateFormatter and its default implementation DefaultDateFormatter have been added. This service is a complete re-implementation of the date/time parsing and formatting logic in the BCL (i.e. DateTime.Parse() and DateTime.ToString()). This implementation is more flexible and configurable, is free from tightly coupled dependencies on the local machine configuration, supports arbitrary combinations of culture and calendar, and does not contain the subtle bugs that unfortunately exist in the BCL implementation. Except for the bugs in the BCL, this implementation is otherwise 100% compatible with the BCL implementations for all possible cultures and formats, and comes with a full suite of unit tests to prove it.

The IDateTimeFormatProvider abstraction and its implementations have been extended with more formats.

The property Display of type DateTimeFieldDisplays has been added to the DateTimeField class, allowing consuming code to determine whether the field is configured in date-only, time-only or date/time mode and interpret the DateTime value accordingly.
Aug 8 at 10:01 AM
Great work. @Decorum, Thank you very much for the time which you are spending on this important feature. I am interested to test these changes as soon as possible.
Developer
Aug 8 at 1:53 PM
Thanks Mehran, my pleasure. Can you help with this?
https://orchard.codeplex.com/discussions/559954
Developer
Aug 8 at 3:24 PM
Hey Daniel, Awesome changes dude!

Otta interest, did you upgrade to the latest cut of the http://keith-wood.name/calendars.html library, which is at 2.0.0?

N
Developer
Aug 8 at 4:05 PM
Thanks Nick - no, I didn't. When I started the work I looked if something newer was available, but back then nothing was. Seems 2.0.0 was released since. Maybe it would be good to also do that upgrade in time for 1.9... something you'd be interested in taking on?
Developer
Aug 8 at 6:29 PM
Edited Aug 8 at 6:29 PM
All done, I updated TimeEntry to 2.0.1 and Calenders to 2.0.0 - Also I updated TinyMCe to have RTL based on the current culture, same process you did for calendars.
Coordinator
Aug 8 at 8:16 PM
works great for me, good job
Developer
Aug 8 at 8:43 PM
Looks great Nick, good job! Just one thing - it looks like you also changed in the legacy shape templates for the old jQuery UI date picker - was that intentional?
Developer
Aug 8 at 8:56 PM
Nick, one more thing. I don't think the RTL change for TinyMCE has the effect you were hoping for. It seems to affect the content being edited, not the UI of the editor. So now when editing English text, it is all right-aligned.
Developer
Aug 8 at 9:21 PM
TinyMCE: Yeah, this was the only thing I could see in TinyMCE that would provide RTL. But if you are editing in a non RTL language, then its LTR. The way to make the editor change is to change the HTML attribute to TRL too... - that's to come!

Legacy Shape Templates: No - that as not, Erm, where did I do this? I will revert that part of the change.

Thanks for looking for me.
Developer
Aug 8 at 9:48 PM
TinyMCE: I think we need to revert this one too. The RTL of the content area should be based on the language of the content you are editing, not the configured culture of the site. Those are very often not the same thing.

Templates: DatePickerLocalization.cshtml and TimePickerLocalization.cshtml. Those are not used anymore, only retained for any modules or themes that might still use the jQuery UI pickers directly.
Developer
Aug 8 at 9:53 PM
TinyMCE: Ahhhhh - I understand the issue with this - I will fix it in the morning.

Templates: I have reverted those two files. Thanks for pointing out.

regional was renamed to regionalOptions in the new Date stuff.


Developer
Aug 8 at 9:56 PM
Do you think the DatePicker RTL should be based on the culture of the content you are editing too?


Developer
Aug 8 at 10:04 PM
No. For that case I think it makes sense to configure it based on the culture of the site, because it's only a part of admin UI, and it's reasonable to assume all parts of the admin UI are localized. The same goes for the UI of TinyMCE (buttons, toolbar, chrome, etc) - I wouldn't mind those being RTL at all. It's only the content area I mind, because that's not about displaying admin UI, but rather about displaying pure content, the language of which we can't assume based on the configured culture.
Developer
Aug 8 at 10:18 PM
The content area needs to be RTL based on the selected culture of that content item, right?


Developer
Aug 8 at 10:24 PM
That would make a lot of sense, yes.
Developer
Aug 8 at 10:26 PM
Okay cool - I will get that fixed.


Developer
Aug 8 at 10:31 PM
So the rules will be...

If the content item does not have a ILocalizationAspect, then the culture will always be run off of the Site Culture.

If the content item does have a ILocalizationAspect, then the culture is that of the ILocalizationAspect

For new content where you have a master content item and an ILocalizationAspect, then is based on the selected culture of the content item you are creating.

For new content where you do not have a master content item then is based on the selected culture of the content item you are creating.

I think that makes sense.


Developer
Aug 8 at 11:06 PM
Hmm... my only hesitation is setting content RTL based on site culture in absense of an ILocalizationAspect. I'm torn honestly. But I guess it comes down to which choice would produce the desired result most often. For example, if you don't enable content localization at all, and you set the site culture to fa-IR, should it be assumed that your content is in Farsi and hence rendered RTL? It's reasonable perhaps. If you have a site where this is not the case, it might be reasonable that you have to enable content localization and set the culture of your content to en-US or some other LTR culture. So, long story short... yes, I guess I agree. :)
Aug 8 at 11:57 PM
Edited Aug 8 at 11:59 PM
Strange that tinyMCE does not switch automatically the rtl/ltr mode according the current language code ?
Developer
Aug 9 at 12:23 PM
@Decorum - Okay, fixed and pushed. I have added a new extension as Directionality can be used on any HTML element, so making it common makes sense. just returning rtl or ltr.

@CSADNT - Well, its not that simple, Orchard sets the current culture as a property in Orchard, not the thread. So something would still need to tell TinyMCE about that. The other issue is that as @Decorum pointed out, The Current/Site Culture does not dictate if the content you are editing is RTL/LTR. What dictate that is the ILocalizationAspect attached to the content, and if that not available, then its Site Culture. You would never want to edit a content item with the current culture as you could mess up a RTL Content Item when your current culture is LTR.


Aug 9 at 1:35 PM
Edited Aug 9 at 1:38 PM
Hi Nick,

Thanks for answer, but from what I see here, tinymce language packs automatically contain and set the direction, I d'ont understand why we should bother setting it ?
http://www.tinymce.com/forum/viewtopic.php?id=32748

the important part is here
@beamish: It's now moved to language packs. The language files needs to have a _dir: 'rtl' property to force RTL layout. Out translation service will automatically add this property for the language pack. You can manually force RTL for development purposes by setting tinymce.i18n.rtl = true; before tinymce.init.
Eventually we could add the 2 buttons to force it as explained here
http://www.tinymce.com/wiki.php/Plugin:directionality


And concerning your algorithm to set the default language, I ok to use ILocalizationAspect / LocalizationPart when present, but when there is no localization, we should not directly fall back to the site but use the Language selectors and localization manager to decide which culture to use.

@decorum: looks fine
Developer
Aug 9 at 2:28 PM
CSADNT: And how do the language packs get activated? As far as I can see, we are not doing that. TinyMCE can't just guess the culture of the site, nor of the content being edited, and somehow magically activate the correct language pack.

However, the idea of adding buttons to manually control RTL makes a lot of sense to me - didn't know that could be done. But I still think it should default to the site culture when there is no content-specific locale.
Aug 9 at 2:43 PM
Edited Aug 9 at 4:32 PM
Decorum wrote:
CSADNT: And how do the language packs get activated? As far as I can see, we are not doing that. TinyMCE can't just guess the culture of the site, nor of the content being edited, and somehow magically activate the correct language pack.
I see the irony of your magically :)
Isn't it in the tinyMCE.init if this method still exists in v4
But I still think it should default to the site culture when there is no content-specific locale.
Why I don't understand ?
if you use a good localization module everything will be ok using the normal Orchard process ? And you will expect tinymce to use the language rules you have set.
When I wrote this one, https://bitbucket.org/csurieux/datwendo.localization , one of my concerns has been tinymce but at that time it was not actual code.
Developer
Aug 9 at 3:45 PM
I also like the idea of adding the buttons... maybe if you have a site that supports RTL and LTR, then add the buttons, else default to Site Culture and don't add the buttons - like it.

The language pack's I considered adding anyway's, the problem is that you cant just plug them in as TinyMCE will 404 if one doesn't exist, there is no fall-back. (Let me have a think about this).
Aug 9 at 4:36 PM
if it 404, it is a simple admin and setup pb easy to fix, considering we always limit cultures to the ones declared by admin in orchard settings
Aug 9 at 5:11 PM
Well, if there is no localization for a specific culture in TinyMCE then there is no "easy fix".

@Nick: I did something similar. That works for site culture but doesn't take cotent item culture into account.
Aug 9 at 9:19 PM
Edited Aug 9 at 9:23 PM
@JasperD looks good because it calls cultureManager.
But the real problem is when you create a localization for an existing item, Tinymce is called with the culture from cultureManager where it should take the culture for the target localization. There should be a script in the localization editor used by Translate controller method, just as what is done for the default route to include the culture, which overrides tinymce culture for the target culture
Nov 9 at 10:12 AM
Very useful changes. Thank you Decorum. I'm waiting for 1.9.