[Pharo-project] zip bug [IMPORTANT] probably related to the recent Zip changes
Sean could you have a look? Because when I tried to click on Balloon-CamilloBruni.85.mcz in PharoInbox I get a month as 13 asDateFromZip "See Date>>zipped comment" | yearsSinceDosEpoch month day year | yearsSinceDosEpoch := self >> 9. "High 7 bits" month := self >> 5 & 2r1111. "Middle 4 bits" day := self & 2r11111. "Low 5 bits" year := DateAndTime dosEpoch year + yearsSinceDosEpoch. ^ Date year: year month: month day: day.
Probably related to Name: Compression-SeanDeNigris.97 Author: SeanDeNigris Time: 21 May 2012, 4:19:27.939 pm UUID: 4936c6ef-dc79-4a72-a4c7-13b8a6840d04 Ancestors: Compression-MarcusDenker.95 * Fix conversion to/from DateAndTime and DOS timestamps. DOS format per http://blogs.msdn.com/b/oldnewthing/archive/2003/09/05/54806.aspx and http://mindprod.com/jgloss/zip.html * Update tests, removing #testAsDosTimeIndependentOfTimezone (they are not independent; DOS timestamps are in local time) * Renamed DateAndTime>>asDosTime to #asDosTimestamp (it is not a time, but an encoded date/time) * Renamed DateAndTime class>>fromDosSeconds: to #fromDosTimestamp (it is not a number of seconds, like a UNIX or Squeak date/time) * Updated ZipArchiveMember to use new method names On Jun 10, 2012, at 11:03 PM, Stéphane Ducasse wrote:
Sean could you have a look?
Because when I tried to click on Balloon-CamilloBruni.85.mcz in PharoInbox
I get a month as 13
asDateFromZip "See Date>>zipped comment"
| yearsSinceDosEpoch month day year | yearsSinceDosEpoch := self >> 9. "High 7 bits" month := self >> 5 & 2r1111. "Middle 4 bits" day := self & 2r11111. "Low 5 bits" year := DateAndTime dosEpoch year + yearsSinceDosEpoch. ^ Date year: year month: month day: day.
Stéphane Ducasse wrote
Sean could you have a look?
Because when I tried to click on Balloon-CamilloBruni.85.mcz in PharoInbox
I get a month as 13
I'll take a look. I'm very confident about that code, because I tested it over and over with real data from different sources. Although, it makes sense that, if we've been saving files with wrong values and loading them using the same formula, we wouldn't know they were wrong until we fix the format... If that's the case, maybe wrap the unzip with a check that defaults to the old format if the date is invalid or strange (e.g. year 1962)... Sean -- View this message in context: http://forum.world.st/zip-bug-IMPORTANT-probably-related-to-the-recent-Zip-c... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On Mon, Jun 11, 2012 at 2:58 AM, Sean P. DeNigris <sean@clipperadams.com>wrote: ... If that's the case, maybe wrap the unzip with a check that
defaults to the old format if the date is invalid or strange (e.g. year 1962)...
What's strange about 1962? I happen to think it's a rather nice year ;-) -- Cheers, Peter
Stéphane Ducasse wrote
Sean could you have a look?
I can't get latest 2.0 bc Jenkins is down. Can you send me an image? Otherwise I'll load the slice into 20118 Stéphane Ducasse wrote
Because when I tried to click on Balloon-CamilloBruni.85.mcz in PharoInbox
Where did you click on it? I need steps to reproduce... -- View this message in context: http://forum.world.st/zip-bug-IMPORTANT-probably-related-to-the-recent-Zip-c... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Sean P. DeNigris wrote
Where did you click on it? I need steps to reproduce...
Okay, I tried to merge it and got the error when the package was being read... I did some experiments... * create a zip at Mac command line -> new Pharo code correctly parses date/times * unzip Balloon-CamilloBruni.85.mcz at Mac command line -> Mac reports date/time as 1/23/2010... I'm assuming this is wrong because pre-fix, Pharo reported 29 September 2011 So, I think the code is correct, but transitioning is a problem... let me see write up a fallback defaulting to the old date format for invalid dates... -- View this message in context: http://forum.world.st/zip-bug-IMPORTANT-probably-related-to-the-recent-Zip-c... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Sean P. DeNigris wrote
let me see write up a fallback defaulting to the old date format for invalid dates...
Damn!! I'm screwed because the bug we found yesterday in #ensureDirectory (in the VM I think) makes all my tests fail, so I'm refactoring blind!! -- View this message in context: http://forum.world.st/zip-bug-IMPORTANT-probably-related-to-the-recent-Zip-c... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Okay, here is an issue: Issue 6054: Error when reading zip files with Squeak date/time format http://code.google.com/p/pharo/issues/detail?id=6054 with a fix attached and a comment pointing back here:
Attached are two cs files that have a temporary fix enabling us to load mcz and other zip files, even if they have the old, incorrect date format. They should be refactored and there is a larger question to discuss [1], but they work under manual testing.
The new behavior when reading zip file timestamps is: * parse as a DOS timestamp * if an error is encountered, fall back and treat it like a Squeak (i.e. incorrect) DOS timestamp. This may return an incorrect result, but will not cause an error
The changes were tested by hand on Balloon-CamilloBruni.85.mcz, which Steph was having a problem with because there was 13 as a month index, which blew up [1]
The larger issue I was thinking about is: should we extend this fallback to all DOS timestamp parsing, but now that I think about it: * I imagine there's not much of that going on besides zipping, lol * I think it'd be better to signal an InvalidDosTimestamp, describing the change, and pointing users to how to convert their data, instead of silently trying to convert What do you think? Cheers, Sean -- View this message in context: http://forum.world.st/zip-bug-IMPORTANT-probably-related-to-the-recent-Zip-c... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Sean P. DeNigris wrote:
Sean P. DeNigris wrote
Where did you click on it? I need steps to reproduce...
Okay, I tried to merge it and got the error when the package was being read...
I did some experiments... * create a zip at Mac command line -> new Pharo code correctly parses date/times * unzip Balloon-CamilloBruni.85.mcz at Mac command line -> Mac reports date/time as 1/23/2010... I'm assuming this is wrong because pre-fix, Pharo reported 29 September 2011
So, I think the code is correct, but transitioning is a problem... let me see write up a fallback defaulting to the old date format for invalid dates...
-- View this message in context: http://forum.world.st/zip-bug-IMPORTANT-probably-related-to-the-recent-Zip-c... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
What do you get if you unzip it outside Pharo? from the operating system?
Ben Coman wrote
What do you get if you unzip it outside Pharo? from the operating system?
Sean P. DeNigris wrote
* unzip Balloon-CamilloBruni.85.mcz at Mac command line -> Mac reports date/time as 1/23/2010... I'm assuming this is wrong because pre-fix, Pharo reported 29 September 2011
So Mac's zip came up with a valid, but bogus date; while Pharo raised an error... -- View this message in context: http://forum.world.st/zip-bug-IMPORTANT-probably-related-to-the-recent-Zip-c... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
participants (4)
-
Ben Coman -
Peter Hugosson-Miller -
Sean P. DeNigris -
Stéphane Ducasse