Help needed from native Persian/Farsi speakers - correct date/time formats for Persian calendar?

Topics: Localization
Developer
Aug 5, 2014 at 8:21 PM
Edited Aug 5, 2014 at 8:59 PM
Hello all Iranians out there! ;)

For the upcoming 1.9 release (hopefully) I have decided to once and for all adress all of the problems and shortcomings with the custom calendar support that was introduced in 1.8. In order to fix some of the issues, I have had to completely restructure the underlying date formatting/parsing logic in Orchard.

This restructuring has opened up an interesting possibility: it is now possible to implement, in a clean and robust way, support for proper names and formats for the Persian calendar when used in combination with the fa-IR (Iranian Farsi) culture, something a lot of people have requested. The .NET Framework does not contain these localizations because for some strange (probably political) reason, the PersianCalendar is not one of the optional calendars for the fa-IR culture, or any other for that matter. This causes many issues. For example, currently when using the fa-IR culture in combination with the Persian calendar, the Farsi month names for the Gregorian calendar are displayed instead of the proper Farsi month names for the Persian calendar.

However, these is new hope - after the refactorings I now have a good clean way to add "hard-coded" (in a good way) support for the correct names and formats to Orchard, without forcing users to resort to .PO files to add them.

Given that the Persian calendar is the only calendar to be "orphaned" (i.e. not supported for any culture!) in the .NET Framework, something generally considered a serious bug, I think it's justified to add this particular override to Orchard as long as it can be done cleanly.

To accomplish this, I just need some help providing me with the correct information. To that end, I have put together a template document where I would like someone with native Farsi skills to provide the correct and officially recognized names and supported formats for the Persian calendar in the fa-IR culture.

Please see the attached Word document here:
https://www.dropbox.com/s/w7jo0dzxz9e8mhb/DateTimeFormatInfo%20for%20PersianCalendar.docx

If someone could help fill out the required information and email the completed document to me, I can try to get this into the 1.9 release.

Thanks in advance! I think this is something that would make Orchard easier to use for all Iranian users, and help increase international adoption of Orchard.

Notes:
  • The en-US and fa-IR formats for Gregorian calendar as provided by the .NET Framework are provided as examples for reference in the first two columns.
  • The fa-IR formats for Persian calendar should be added in the third column.
  • The first bold item in a list of supported formats is considered the standard format, and used by default when printing. The rest are supported when parsing.
  • For a reference on what the individual tokens/specifiers in a format pattern mean, see http://msdn.microsoft.com/en-us/library/8kb3ddd4(v=vs.110).aspx
  • For a reference on the different pattern categories (left-most column), see http://msdn.microsoft.com/en-us/library/az4se3k1(v=vs.110).aspx
  • The patterns in the fa-IR columns should be in reverse order from how they are actually printed and read, because fa-IR is a right-to-left culture. There is a screen shot in the document illustrating this.
Aug 10, 2014 at 3:40 AM
Thank you so much for your works. I will fill the document and will send to you as soon as possible.
Aug 10, 2014 at 5:24 AM
I'm ready to help too. Of course, @Arman is my professor and i prefer that he determines the best format.
Aug 10, 2014 at 5:35 AM
@Mehran, Thank you. I will send the document for you also. Please feel free to correct if you see some mistakes.
Aug 12, 2014 at 8:21 AM
@Decorum, I sent the document for you.
@mehranrezaei, I sent it for you too.
Developer
Aug 12, 2014 at 1:50 PM
@Arman, thank you so much for this - it's exactly what I needed! Thank you also for being so thorough with the notes, it really helps me understand the details. I will do my best to incorporate these names and formats, with all the details you provided. I'll let you know if I have any questions.
Aug 12, 2014 at 2:21 PM
Edited Aug 26, 2014 at 7:45 PM
@Decorum, Special thanks to you because you are spending large time on this.
Please feel free to contact me if you need more assistance.
Aug 12, 2014 at 4:57 PM
Edited Aug 12, 2014 at 5:01 PM
@Decorum, As Mehran mentioned we thank you for your great works. Calendar is very important feature of a CMS. If you have any more questions about Persian Calendar, I'm ready to help you.
Mehran, Thank you for verifying the document.
Developer
Aug 15, 2014 at 10:19 PM
@Arman, I just need to verify that you have really understood the RTL aspect of the formats correctly, because the strings you have entered in look inconsistent to me. As I explained, even though the culture is RTL the format strings are still LTR, but when they are printed they are reversed.

For example:

For fa-IR Gregorian the .NET short date format string is "dd/MM/yyyy" which will be printed as "2014/05/31" in RTL.
For fa-IR Persian as the corresponding format you have specified "yyyy/MM/dd" which will be printed as "31/05/2014" in RTL.

Is this your intention? Is it really your opinion that the .NET formats for fa-IR Gregorian should be completely reversed just because we change to the Persian calendar?
Developer
Aug 15, 2014 at 11:18 PM
@Arman, by the way I decided to use the separator as you wished, but in addition I also added the corresponding formats with a simple space separator so that they are available for parsing. Like so:
persianFormats.SetAllDateTimePatterns(new[] {
    "dddd، d MMMM yyyy ساعت HH:mm:ss",
    "dddd، d MMMM yyyy ساعت H:mm:ss",
    "dddd، d MMMM yyyy ساعت h:mm:ss tt",
    "dddd، d MMMM yyyy ساعت hh:mm:ss tt",
    "d MMMM yyyy ساعت h:mm:ss tt",
    "d MMMM yyyy ساعت hh:mm:ss tt",
    "d MMMM yyyy ساعت HH:mm:ss",
    "d MMMM yyyy ساعت H:mm:ss",
    "dddd، d MMMM yyyy HH:mm:ss",
    "dddd، d MMMM yyyy H:mm:ss",
    "dddd، d MMMM yyyy h:mm:ss tt",
    "dddd، d MMMM yyyy hh:mm:ss tt",
    "d MMMM yyyy h:mm:ss tt",
    "d MMMM yyyy hh:mm:ss tt",
    "d MMMM yyyy HH:mm:ss",
    "d MMMM yyyy H:mm:ss",
}, 'F');
Does that make sense?
Developer
Aug 16, 2014 at 12:12 AM
Edited Aug 16, 2014 at 12:19 AM
@Arman, unfortunately I spoke too quickly. The separator "ساعت" could not be added. There is much less flexibility than I thought. In fact, we can only specify the date, time and year/month formats separately; all the combined date/time formats are inferred by concatenating the separate formats together using a hard-coded space as a separator.

You can see this here in the comment above the SetAllDateTimePatterns method here.

Anyway, I don't think it matters much, because the combined date/time formats you had specified were actually only permutations of the separate formats. The only thing we actually lose is that it's not possible to use the "ساعت" separator - instead a space will always be used.

Other that that, I have incorporated all of your specifications, including the proper "،" character instead of comma. Everything has been pushed to the 1.x branch. Please try it out as soon as you get a chance, while we still have time to make adjustments before the 1.9 release.

One additional question: are your formats applicable only to Iran, or also other locales where Farsi is used (i.e. Afghanistan and Tajikistan) in combination with the Persian calendar?
Aug 18, 2014 at 11:38 AM
Edited Aug 18, 2014 at 11:45 AM
@Decorum, Excuse me for my delay. I just saw your message now.
I know about RTL aspect of the formats and i think my formats are correct and .NET formats are not.
  1. Please open Notepad, MS Word or any other editor and write any in "yyyy/MM/dd" format. Then change direction to RTL (press right Ctrl and right Shift).
    The value remains in the correct format and does not change to "dd/MM/yyyy" format.
  2. I tested all of the formats that i sent to you. you can see the screenshot:
Image

I don't agree that .NET short date format ("dd/MM/yyyy") will be printed as "2014/05/31" in RTL. you can see the screenshot. RTL direction does not change order of numbers because numbers are written from left to right even in RTL languages.

Also i don't agree that my format ("yyyy/MM/dd") will be printed as "31/05/2014" in RTL. you can see the screenshot again.

Yes, I think .NET formats should be completely changed to my formats.
  1. If there is no way to use "ساعت" and the space separator is hard coded, forget it simply. no problem. This is the reason that I included this separator as note and i didn't include it in main table. thank you for your notice.
  2. Sure, I will test 1.x branch as soon as possible.
  3. Personally, i think my formats applicable to all locales where Farsi is used not only to Iran.
At the end, thank you again for spending your valuable time.
Developer
Aug 19, 2014 at 12:24 AM
@Arman, you are correct. I was wrong. I have tested more thoroughly now. The formatter will only output RTL for formats which include literal text of any kind, i.e. anything except standard separators and specifiers. This includes the standard specifiers D, f, F, m, y and U, as well as any custom formats strings you create that contain spaces or literal text. I think all is good then.
Aug 19, 2014 at 11:15 AM
@Decorum Thank you for your notice. I tested 1.x branch today. You can see the results as the following steps.

Step 1. I did fresh install of 1.x branch (2ef4eea1d79aef6a3534934c613aac94e2c55928).

Step 2. I tried to edit the default homepage of Orchard in Dashboard. The current date in Created On field seems as M/d/yyyy format and Date picker can select in the same format that this format is correct for current culture calendar (Gregorian Calendar). In front-end the published date seems as (Tuesday, August 19, 2014 11:56:10 AM) that is correct.

Step 3. In Dashboard > Settings, I added fa-IR culture and i selected it as default culture. (The Default Site Calendar remains as Culture calendar.)
In this case we have the following formats from the default implementation of calendar in .NET that are not correct for fa-IR culture. (dot NET only provides some wrong localization on the names of months, weekdays, ... with wrong format with the values of Gregorian calendar.)
  • In front-end: (سه شنبه, 19 اوت 2014 11:56:10 ق.ظ) is not acceptable as fa-IR format.
  • In Create On field in Dashboard: (Date: 19/08/2014 = dd/MM/yyyy and Time: 11:56:10 ق.ظ) are not acceptable.
  • Date picker selects in dd/MM/yyyy format that is not correct.
Step 4. In Dashboard > Settings, I restored the current culture to en-US and i changed the Default Site Calendar to Persian calendar.
  • In front-end: (Tuesday, May 28, 1393 11:56:00 AM) is not acceptable as Persian Calendar format.
  • In Create On field in Dashboard: (Date: 5/28/1393 = M/d/yyyy and Time: 11:56:00 AM) are not acceptable for Persian calendar.
  • Date picker selects in M/d/yyyy format that is not correct for Persian calendar. Also Date picker (like front-end) shows Persian year with English month name.
I understand calendar system and localization are different things but i think at least when we select Culture Calendar option, the correct calendar for that culture must be used. For example culture calendar for the fa-IR must be Persian Calendar.

Step 5. Main test: I selected fa-IR for the current culture and Persian Calendar for the Default Site Calendar. In this case firstly i saw (سه شنبه, 28 مرداد 1393 11:56:00 ق.ظ) at front-end that is not correct. BUT I restarted IIS Express and the result is as the following
NOTE: It seems that restart is needed only when my tests have orders as the above steps. For example if you select fa-IR and Persian Calendar immediately after fresh install restart is not needed!
  • In front-end: (سه شنبه، 28 مرداد 1393 11:56:00) is perfect! I like it. (Of course it can be better if you use Persian digits for view but it is not very important. It may be done by view template overriding.)
  • In Create On field in Dashboard: (Date: 1393/05/28 = yyyy/MM/dd and Time: 11:56 = 24-hour) is perfect.
  • Date picker seems good and it is tolerable localization except it uses some Arabic character such as (ي). This is not very important. It is internal problem of date picker. It needs better localization file for fa-IR.
  • The main problem is: Date picker selects in dd/MM/yyyy format that is not correct for Persian calendar. if you select a date by Date picker then click on publish button you get error '28/05/1393 11:56' could not be parsed as a valid date and time.
I hope you find these tests helpful.
Developer
Aug 19, 2014 at 3:57 PM
@Arman, thank you for testing!

Step 3: I know you find these formats unacceptable, but this was not the problem I intended to solve. I only set out to adress the combination of fa-IR and PersianCalendar, which was much more wrong because of the incorrect month names.

Step 4: Here I simply disagree with your opinion. You say "are not acceptable formats for Persian calendar", but the formats have nothing to do with the calendar, and everything to do with the selected culture. If you configure the site with en-US culture, those formats should and will be used, regardless of what calendar you have selected. This is correct behavior.

Step 5: I will investigate the problem with the date picker outputting the wrong format. This should not happen.
Aug 19, 2014 at 4:17 PM
As i mentioned above with label "Main test", I know your target is about step 5.
Also i don't agree with you about step 4. Persian Calendar means Persian Calendar. Or we must remove Persian Calendar option when selected culture is not fa-IR or if the user can select Persian Calendar the user must see Persian Calendar concepts. Why year changes to 1393 but month does not?
The combination of 1393 and August has not meaning.
Developer
Aug 19, 2014 at 4:27 PM
Yes, but your complaint in step 4 was not only about the month name August, but also the formats. The formats are completely unrelated to the selected calendar. Can we agree on that at least?

Regarding the month names, I fully understand this and I agree with you on principle, but this is simply not the way it is structured in the .NET Framework. The calendar itself is only a representation of how the dates are calculated, how many days in a given month, months in a given year, etc. A calendar cannot and does not contain any names for months or days. Instead, the month name is a function of the combination of the selected culture and the selected calendar.

To prove my point: If you select en-US in combination with PersianCalendar, what would you expect to be printed instead of August? We can't print "آبان" because that's not English! And even if there were an English name for that month, if we provide an override for it, why should be not then also provide localizations for every other selectable culture, Swedish, French, Italian, etc? This is simply not realistic to do.
Aug 19, 2014 at 5:44 PM
I understand you and my talk was about month names. Yes, format was wrong word.
But I don't believe that we can summarize a Calendar in your definition. Also the problem is not only in name of the month. Please be aware that the problem of step 4 can show Tuesday, Feb 31, 1393 11:56:00 AM to the user. This is completely wrong. This is not a valid date in Persian, Gregorian or other calendar system.
Also if the user can select Persian Calendar in en-US culture, Tuesday, Aban 30, 1393 11:56:00 AM is better than Tuesday, May 28, 1393 11:56:00 AM or Tuesday, آبان 28, 1393 11:56:00 AM and if providing localization is difficult, OK, I think better option is we prevent the user from selecting unsuitable calendar for a culture.

Anyway i think this is not very important to take your time. Main target is Persian Calendar and fa-IR work together correctly. The only remaining problem as i said before is outputting format of Date picker. Now Orchard has great calendar feature. Thanks a lot.

Thank you so much for your patience.
Developer
Aug 20, 2014 at 12:22 AM
@Arman, I have fixed the bug in step 5 now and pushed to the 1.x branch. Please test again if you can to verify.
Aug 21, 2014 at 7:49 AM
Congratulations to all the Persian users of Orchard. Now there is Persian calendar in Orchard out of the box.
@Decorum, Thank you so much on behalf of all Persian users of Orchard. You've spent a lot of time on this feature. I tested it. It is Perfect.
Developer
Aug 21, 2014 at 1:13 PM
Thank you @Arman that's really nice to hear! You help in this was invaluable.
Coordinator
Aug 21, 2014 at 6:10 PM
Calendars, done
RTL admin, done, have you seen it Arman ?

Now we just need a full translation
And maybe it could deserve a blog post, in Persian !
Aug 21, 2014 at 6:22 PM
Ooh, RTL admin? Really? Can you please get me a link to its discussion?
Developer
Aug 21, 2014 at 6:27 PM
@Arman: The discussion started further down in this thread:
https://orchard.codeplex.com/discussions/560347

The changes are in the branch "feature/localizationchanges", not yet merged into 1.x. Nicholas Mayne is working on them.
Aug 21, 2014 at 6:36 PM
@Decorum, Thanks a lot. I'm really happy for this news. I'm going to test it.
Nov 9, 2014 at 10:10 AM
I see a lot of contribution of Persian speakers to my favorite CSM. Many thank to all of you and many thanks to people bringing them in the Orchard CMS.
Dec 29, 2014 at 7:50 AM
if you want numbers to be Persian you can change the theme to rtl one. it automatically change the numbers. but i have another question.
how did you change (سه شنبه, 19 اوت 2014 11:56:10 ق.ظ) to ( سه شنبه, 28 مرداد 1393 11:56:00 ق.ظ)?
Developer
Dec 30, 2014 at 1:17 PM
maryam_karimi wrote:
if you want numbers to be Persian you can change the theme to rtl one. it automatically change the numbers. but i have another question.
how did you change (سه شنبه, 19 اوت 2014 11:56:10 ق.ظ) to ( سه شنبه, 28 مرداد 1393 11:56:00 ق.ظ)?
I'm not sure I understand the question. Are you asking how I implemented support for the Persian calendar?
Dec 30, 2014 at 2:33 PM
Hi Decorum
I mean how can I use shamsi (like in persian calendar) date with shamsi months?
thanks a lot
Dec 30, 2014 at 2:39 PM
Hello Maryam

If you are using 1.x branch, Follow the following steps:
  1. Add fa-IR culture and select it as default.
  2. Set Persian Calendar in site settings page.
Developer
Dec 30, 2014 at 2:40 PM
Maryam,
Since my commit on Aug 5 this should work by default in the 1.x branch. What version did you try?
Dec 30, 2014 at 2:45 PM
Orchard 1.8.1 is the version I use. I exactly do what you said but it still show it like:

جمعه، 4 اُكتبر 1393 9:20:00 ق.ظ
Developer
Dec 30, 2014 at 2:50 PM
Maryam,
I fixed this after 1.8.1. You have to use a later version. Try the latest in 1.x branch, or wait for 1.9 to be released.
Dec 30, 2014 at 2:51 PM
As @Decorum mentioned, this feature has been fixed in 1.x branch. You can wait for next (1.9) Orchard release or use 1.x branch.
Dec 30, 2014 at 2:53 PM
Edited Dec 30, 2014 at 3:00 PM
Thank you very much. Ok, so I should update it.if I update the version, the content and other configuration will remain the same and with out changes?
Dec 30, 2014 at 3:19 PM
Edited Dec 30, 2014 at 3:23 PM
@Decorum, I see this date "يكشنبه، 7 دي 1393 10:35:41" in my test today. Please be aware that "ي" is not Persian character. It shouldn't be used in Persian for month or day names. I listed wrong characters that shouldn't be used in Persian Calendar:

"ي" is not correct. It must change to "ی"
"ك" is not correct. It must change to "ک"

It is better that you find and replace these characters in calendar strings if you have time.
Developer
Dec 30, 2014 at 3:36 PM
Arman,
Can you please create an issue, and give me the resulting issue number, and I will correct it.
Developer
Dec 30, 2014 at 4:02 PM
Arman,
  1. I assume the "ي" character should only be changed in its standalone form in the "دي" month name, not in its concatenated form in "فررودين" for example?
  2. I can't find any instance of the "ك" character standalone, nor in day names or month names. What exactly should change? Do you mean the concatenated character in "يكشنبه"?
Developer
Dec 30, 2014 at 7:04 PM
Never mind, I figured it out. Pushed to 1.x now.
Dec 30, 2014 at 7:55 PM
@Decorum, Excuse me for delay.
  1. Please note:
    "ي" with code (U+064A) is Arabic Letter Yeh. It has also initial and medial form ("ﻳ", "ﻴ")
    "ی" with code (U+06CC) is Farsi Letter Yeh. It has also initial and medial form ("ﯾ", "ﯿ")
    Writing of these characters are similar to each other in initial or medial form. But they have different codes.
    In the same way there are different codes for "ك" and "ک". Initial and medial form are similar to each other.
    The correct way is replacing these characters in any forms and any places because as i mentioned above they have different codes.
    Therefore simplest way is that you use find and replace dialog of your editor (Notepad, VS, ... ). You can type "ي" and replace it with "ی". Also "ك" with "ک" in any places.
  2. No matter what is the place of character. Also the appearance of the letters does not matter. The important thing is that they have a different code. Therefore i suggest find and replace in any places. Yes, "ك" must be replaced in "یکشنبه" or any other words.
Developer
Dec 30, 2014 at 7:58 PM
Arman,
Yes this is what I concluded also, and this is what I committed in 1.x. However, it is not as simple as a search/replace, because the day names were not overridden at all - the .NET Framework day names were used. However I added overrides for the day names also, like so:
var persianDayNames = new[] {
    "یکشنبه", // Changes the Arabic "ي" and "ك" to the Farsi "ی" and "ک" respectively (incorrect in .NET Framework).
    "دوشنبه",
    "سه شنبه",
    "چهارشنبه",
    "پنجشنبه",
    "جمعه",
    "شنبه"
};

persianFormats.DayNames =
    persianFormats.AbbreviatedDayNames =
    persianDayNames;
Dec 30, 2014 at 8:04 PM
Edited Dec 30, 2014 at 8:06 PM
I confirm that these overrides are correct. There are no Arabic letters.
Thank you so much.
Jan 28, 2015 at 3:49 PM
Edited Jan 31, 2015 at 3:43 PM
Thank you @Decorum for your valuable work also thank you @arman for testing issues
So happy for Rtl admin

Also the good news of 2015 is :
Persian Calendar support also available in recent builds of Windows 10
https://windows.uservoice.com/forums/265757-windows-feature-suggestions/suggestions/6438962-add-persian-calendar-to-windows#{toggle_previous_statuses}