New Blog
Hi all,
I’ve finally made a blog to describe my reversing endeavours. I’m not used to blogging, so I might not write so much.
Anyway, here’s the latest versions of my scripts:
eReader2html.py 0.03 : http://pastebin.com/m28dad336
mobihuff.py 0.03 : http://pastebin.com/mbdcaf3b
mobidedrm.py 0.02 : http://pastebin.com/m12cec25b
February 13, 2008 at 5:40 pm
MobiHuff 0.02 does not work with the Huff dic compressed mobi file at http://www.mobileread.com/forums/attachment.php?attachmentid=10249&d=1202841000
It was generated using mobigen.
mobihuff produced an empty output file
February 14, 2008 at 7:36 am
Try with version 0.03
February 14, 2008 at 4:09 pm
version 0.03 extracts the html successfully, but it doesn’t handle the and tags.
February 14, 2008 at 4:27 pm
Version 0.2 of mobidedrm works fine with huffdic compressed files, but it produces an error with some files that worked fine in v.0.1. I sent you an e-mail with the details.
February 14, 2008 at 4:56 pm
It’s not designed to. It’s just made as a proof of concept to show how the compression algorithm works.
February 15, 2008 at 5:42 pm
I’ve processed well over 200 eReader books with v0.3 and received the following errors:
“Incorrect ereader version (error 2)” This was on 5 files.
“Index Error: String index out of range” This was on 1 file.
Many, many thanks for my huge amount of successful conversions.
February 15, 2008 at 7:27 pm
Thanks much for these scripts! Although I can use DRMed Mobi files on my iLiad, I hate the thought of not being able to read the books I’ve purchased in the future if my reader dies. (although to date the number of books purchased with DRM =1) Baen.com and other places that sell DRM-free books are great.
February 16, 2008 at 9:35 pm
Thanks for the eReader2Html script. It even works with dictionaries, simply they have different id: PDctPPrs.
What is not working: footnotes. Not only they are not tagged in the output file, but even their bodies are missing completely…
February 17, 2008 at 5:16 pm
Any chance of a mobi2html script? It would be nice to be able to get the text out of already DRM-free mobi files.
February 17, 2008 at 6:42 pm
nevermind, found one at: http://www.ida.liu.se/~tompe/mobiperl/
February 22, 2008 at 9:17 pm
Is there a way, short of brute force, to determine if a Mobi or Ereader file is DRMed?
February 25, 2008 at 7:31 pm
I am trying to get mobidedrm to work , unfortunately i am a noob on programming an d terminal. Can anyone geve me some feedback on the error message I got seen below, thnx in advance, Andre awi100_hotmail.com
mobidedrm.py:49: FutureWarning: hex/oct constants > sys.maxint will return positive values in Python 2.4 and up
crc = (~binascii.crc32(s,-1))&0xFFFFFFFF
MobiDeDrm v0.02. Copyright (c) 2008 The Dark Reverser
Traceback (most recent call last):
File “mobidedrm.py”, line 176, in ?
file(outfile, ‘wb’).write(DrmStripper(data_file, pid).getResult())
File “mobidedrm.py”, line 144, in __init__
found_key = self.parseDRM(sect[drm_ptr:drm_ptr+drm_size], drm_count, pid)
File “mobidedrm.py”, line 101, in parseDRM
pid = pid.ljust(16,”)
TypeError: ljust() takes exactly 1 argument (2 given)
February 28, 2008 at 9:28 pm
When running mobodedrm v0.02 I recieve Error: no key found. maybe the PID is incorrect
when I put in a bogus pid I get the error: invalid PID checksum
I would expect an error with a bogus PID, but not with the real PID
February 28, 2008 at 9:30 pm
this is a mobi book (.prc) purchased from mobipocket.com
February 28, 2008 at 10:00 pm
Can these decoder tools work for muliple PIDs? Mobipocket now allows up to 3 (or 4?). Or is it case of running it through the decoder twice?
February 28, 2008 at 10:48 pm
^ Answered my own question ~ just using one PID seems to strip them all or at least opens the file. A quick check shows 3 out of 4 worked fine. The 4th I got the same error as Johnny. Brilliant little utility though ~ I’ve been worried about being stuck with unusable DRM’d books if mobi went belly up!
March 3, 2008 at 10:06 am
Andre: Upgrade your Python installation. From your log, you seem to be running a version pre-2.4, while 2.5 is the current version.
Johnny/Jason: I think that only the first PID in a file works for decryption. (I get “no key found” errors for PIDs not in the file, and “invalid checksum” when it’s in there but not the first one.)
March 3, 2008 at 3:18 pm
How do I get the first PID? I installed mobireader on one PC and when I opened the file I went to the question mark – about and copied the PID from there.
March 4, 2008 at 8:04 am
@Atkinson ~ I thought of that at the time and tried it with both PIDs, neither worked. The vast majority of files do convert fine though, it just seems to be the odd one.
March 4, 2008 at 10:41 am
I;m about 50% way through my collection ~ 212 have converted fine, 73 haven’t. All the ones that fail show basically this series of error messages. The books come from a variety of mobi retailers. Quite often some in the books series, bought at the same time will convert okay. Anyway here’s the error message ~ any help to shed light on what could be the issue would be appreciated
C:\>mobidedrm.py Patterson_1sttoDie.prc C:\zzznodrm\Patterson_1sttoDie.prc 44E6J
JC84G
MobiDeDrm v0.02. Copyright (c) 2008 The Dark Reverser
Decrypting. Please wait…
Traceback (most recent call last):
File “C:\mobidedrm.py”, line 176, in
file(outfile, ‘wb’).write(DrmStripper(data_file, pid).getResult())
File “C:\mobidedrm.py”, line 159, in __init__
extra_size = getSizeOfTrailingDataEntries(data, len(data), extra_data_flags)
File “C:\mobidedrm.py”, line 74, in getSizeOfTrailingDataEntries
num += getSizeOfTrailingDataEntry(ptr, size – num)
File “C:\mobidedrm.py”, line 64, in getSizeOfTrailingDataEntry
v = ord(ptr[size-1])
IndexError: string index out of range
March 4, 2008 at 1:50 pm
my problem was because I was trying to de-code a copy of the original. I had to use the one located in My Documents\My Ebooks
Hope this helps some people
March 4, 2008 at 3:50 pm
Well my problem @10:41 post was cured by using v1 for those which failed v2. I’ve tested all 409 files and every one worked which is pretty brilliant! Now, I’m much happier to buy DRM’d books in the knowledge that I could still access even if mobipocket had a melt down. Thanks Dark Reverser ~ 10/10!
March 9, 2008 at 10:31 pm
Jason (or anyone else) where do I get V1? I am having the same issue.
Thanks Dark Reverser
March 10, 2008 at 7:57 am
Try searching Google for MobiDeDrm v0.01, tell Google to include all results previously excluded, then look at cached copy of page
March 10, 2008 at 1:32 pm
@ atimson
thnx, After upgrading to OSX 10.5 (python is included) the script worked fine, thnx!
March 19, 2008 at 8:24 am
Unfortunately the links to the files on pastebin are reported to be expired or removed!
Is there any mirror out there?
THX in advance
March 20, 2008 at 10:34 am
ereader2html 0.03 has been uploaded at http://pastebin.com/f1fc790cb
March 24, 2008 at 8:46 pm
MobiDeDrm v0.02
March 24, 2008 at 8:48 pm
Trying one more time..
MobiDeDrm v0.02 uploaded at:
http://pastebin.com/m398d265a
March 26, 2008 at 8:58 pm
I’m new to this game so I would appreciate some help.
Can you give me an example of the syntax to use when converting mobipocket files.
The instructions given when I load the script state mobidedrm infile.mobi outfile.mobi PID. How do I point it in the right direction when all my mobipocket files are .prc files.
Thamks in advance.
March 30, 2008 at 5:56 pm
Thanks for the great tools!
Could you repost mobihuff v03? The pastebin entry for it has expired.
Thanks again!
March 31, 2008 at 12:06 am
MobiDeDrm v0.03
March 31, 2008 at 12:07 am
opps v0.02 is the latest, thanks
April 1, 2008 at 3:49 am
Like tdprofffitt said, please repost mobihuff v03 for those of us who missed it.
Thanks!
April 2, 2008 at 6:50 pm
I have a PRC file of a book I bought from BooksOnBoard. The book is no longer available for download. I very much want to be able to read it on my Amazon Kindle. I’ve tried using MOBIDEDRM. (Versions 0.01 and 0.02) I get an error message: “no key found. maybe the PID is incorrect. I’ve verified the PID at BooksOnBoard, and on my Palm T/X. Am I missing a step?
Thanks.
April 5, 2008 at 12:34 am
Both books I used mobihuff on truncated the first 15 or so pages of the book. The cover, TOC, and about half of the first chapter. Both were run through mobidedrm first but that shouldn’t affect the content should it?
Both deDRMed files crashed MobiReader but the DRMed files read fine. Guess I’ll have to stick with LIT books.
April 18, 2008 at 12:00 am
The de drmer is a fantastic tool, I don’t mind buying books but
I hate the idea of not being able to read MY books when my
ereader goes away!
I have a problem with FBReader, a de-DRMed book will frequently
fail to open with an “Unknown Compression Method” error. About
half of my fixed books have this problem…
Any Ideas?
Jerry
April 18, 2008 at 12:02 am
The de drmer is a fantastic tool, I don’t mind buying books but
I hate the idea of not being able to read MY books when my
ereader goes away!
I have a problem with FBReader, a de-DRMed book will frequently
fail to open with an “Unknown Compression Method” error. About
half of my fixed books have this problem…
Any Ideas?
Jerry
April 19, 2008 at 1:28 am
Hi DarkReverser,
I want to thank you for your great work and wish you the very best for the future.
But one last question is on my mind:
MobiDeDRM v0.2 works fine, but if I read the decrypted files with my MobipocketReader, the text (the characters) are replaced with weird signs a/o letters or symbols.
Am I doing something wrong? Or is it a “Python-Error”?
Using:
Python v2.5 (Eng.)
MobiDeDRM v0.1 & v0.2
non-english *.prc-files (like French, Spanish or Sweden)
Any ideas?
So long, and thanks for the scripts …
April 23, 2008 at 10:17 pm
Any working links to mobihuff 0.03 and mobidedrm 0.02?
Please?
April 30, 2008 at 6:55 pm
[...] search (search for MobiDeDrm and Rapid share, for now at least). These nice scripts were developed here. It is easy to use (assuming you can use the command line), you need your PID (as described above) [...]
May 1, 2008 at 9:41 pm
Any working links to mobihuff 0.03 and mobiledrm 0.02 would be greatly appreciated.
Thanks
May 2, 2008 at 1:51 am
You also want MobiDeDRM.py version 0.01 as well. It works in cases where 0.02 does not.
May 12, 2008 at 2:14 pm
Hi DarkReverser,
Would you please repost all three versions of MobiDeDRM as the links say that they have expired.
Thanks
May 13, 2008 at 1:11 pm
[...] do this a script is needed from here. The script is no longer there, but I’ll put the source code up [...]
May 13, 2008 at 9:23 pm
Thanks Mr Reverser (posting this to the right page now)
I get: “Error: no key found. maybe the PID is incorrect”
even on books that “don’t have DRM,” but other tools like mobi2oeb or mobihuff say that the file is DRM encoded.
I think maybe the files are DRM’d to some generic PID that all mobi-browsers test, but I don’t know what this PID might be,
Try the file at:
http://www.lds.org/handheld/pdafiles/pocketpc/MobipocketNewTestamentStudyGuide.zip
May 16, 2008 at 11:04 pm
These scripts as well as the Kindle DRM scripts have been compiled to EXE files and are here:
http://www.demonoid.com/files/details/1479930/775222/
May 22, 2008 at 7:53 pm
Found them: http://rapidshare.com/files/102854939/4DeDRMfiles.zip.html
June 1, 2008 at 2:39 am
[...] New Blog « Darkreverser’s Weblog (tags: drm ebooks convert ereader scripts mobile html) [...]
June 4, 2008 at 5:53 pm
Have tried pastebin for the mobidrm but seems to have expired can it be reposted please
June 6, 2008 at 12:02 pm
liverpool;
it’s here:
http://www.demonoid.com/files/details/1479930/775222/
June 21, 2008 at 5:01 pm
Anyone know where the ereader2html script can be found. Tried searching but cannot find it. Thanks.
June 21, 2008 at 9:11 pm
I’ve had success with Mobidedrm before, now all I’m getting is …
MobiDeDrm v0.02. Copyright (c) 2008 The Dark Reverser
Removes protection from Mobipocket books
Usage mobidedrm infile.mobi outfile.mobi PID”
June 27, 2008 at 8:22 am
Version 0.03
# This is a python script. You need a Python interpreter to run it.
# For example, ActiveState Python, which exists for windows.
#
# Changelog
# 0.01 – Initial version
# 0.02 – Huffdic compressed books were not properly decrypted
# 0.03 – http://www.mobileread.com/forums/showpost.php?p=202684&postcount=76
import sys,struct,binascii
class DrmException(Exception):
pass
#implementation of Pukall Cipher 1
def PC1(key, src, decryption=True):
sum1 = 0;
sum2 = 0;
keyXorVal = 0;
if len(key)!=16:
print “Bad key length!”
return None
wkey = []
for i in xrange(8):
wkey.append(ord(key[i*2])<> 8)) ^ byteXorVal) & 0xFF
if decryption:
keyXorVal = curByte * 257;
for j in xrange(8):
wkey[j] ^= keyXorVal;
dst+=chr(curByte)
return dst
def checksumPid(s):
letters = “ABCDEFGHIJKLMNPQRSTUVWXYZ123456789″
crc = (~binascii.crc32(s,-1))&0xFFFFFFFF
crc = crc ^ (crc >> 16)
res = s
l = len(letters)
for i in (0,1):
b = crc & 0xff
pos = (b // l) ^ (b % l)
res += letters[pos%l]
crc >>= 8
return res
def getSizeOfTrailingDataEntries(ptr, size, flags):
def getSizeOfTrailingDataEntry(ptr, size):
bitpos, result = 0, 0
if size <= 0:
return result
while True:
v = ord(ptr[size-1])
result |= (v & 0×7F) <= 28) or (size == 0):
return result
num = 0
flags >>= 1
# while flags:
if flags & 1:
num += getSizeOfTrailingDataEntry(ptr, size – num)
flags >>= 1
return num
class DrmStripper:
def loadSection(self, section):
if (section + 1 == self.num_sections):
endoff = len(self.data_file)
else:
endoff = self.sections[section + 1][0]
off = self.sections[section][0]
return self.data_file[off:endoff]
def patch(self, off, new):
self.data_file = self.data_file[:off] + new + self.data_file[off+len(new):]
def patchSection(self, section, new, in_off = 0):
if (section + 1 == self.num_sections):
endoff = len(self.data_file)
else:
endoff = self.sections[section + 1][0]
off = self.sections[section][0]
assert off + in_off + len(new) LLLBxxx32s’, data[i*0x30:i*0x30+0x30])
cookie = PC1(temp_key, cookie)
ver,flags,finalkey,expiry,expiry2 = struct.unpack(‘>LL16sLL’, cookie)
if verification == ver and cksum == temp_key_sum and (flags & 0×1F) == 1:
found_key = finalkey
break
return found_key
def __init__(self, data_file, pid):
if checksumPid(pid[0:-2]) != pid:
raise DrmException(“invalid PID checksum”)
pid = pid[0:-2]
self.data_file = data_file
header = data_file[0:72]
if header[0x3C:0x3C+8] != ‘BOOKMOBI’:
raise DrmException(“invalid file format”)
self.num_sections, = struct.unpack(‘>H’, data_file[76:78])
self.sections = []
for i in xrange(self.num_sections):
offset, a1,a2,a3,a4 = struct.unpack(‘>LBBBB’, data_file[78+i*8:78+i*8+8])
flags, val = a1, a2<<16|a3<H’, sect[0x8:0x8+2])
extra_data_flags, = struct.unpack(‘>L’, sect[0xF0:0xF4])
crypto_type, = struct.unpack(‘>H’, sect[0xC:0xC+2])
if crypto_type != 2:
raise DrmException(“invalid encryption type: %d” % crypto_type)
# calculate the keys
drm_ptr, drm_count, drm_size, drm_flags = struct.unpack(‘>LLLL’, sect[0xA8:0xA8+16])
found_key = self.parseDRM(sect[drm_ptr:drm_ptr+drm_size], drm_count, pid)
if not found_key:
raise DrmException(“no key found. maybe the PID is incorrect”)
# kill the drm keys
self.patchSection(0, “” * drm_size, drm_ptr)
# kill the drm pointers
self.patchSection(0, “\xff” * 4 + “” * 12, 0xA8)
# clear the crypto type
self.patchSection(0, “” * 2, 0xC)
# decrypt sections
print “Decrypting. Please wait…”,
for i in xrange(1, records+1):
data = self.loadSection(i)
extra_size = getSizeOfTrailingDataEntries(data, len(data), extra_data_flags)
self.patchSection(i, PC1(found_key, data[0:len(data) - extra_size]))
print “done”
def getResult(self):
return self.data_file
print “MobiDeDrm v0.02. Copyright (c) 2008 The Dark Reverser”
if len(sys.argv)<4:
print “Removes protection from Mobipocket books”
print “Usage:”
print ” mobidedrm infile.mobi outfile.mobi PID”
else:
infile = sys.argv[1]
outfile = sys.argv[2]
pid = sys.argv[3]
data_file = file(infile, ‘rb’).read()
try:
file(outfile, ‘wb’).write(DrmStripper(data_file, pid).getResult())
except DrmException, e:
print “Error: %s” % e
July 4, 2008 at 7:11 pm
Can you repost you version 0.03 with indentation and normal quotation marks, in order to be “paste ready” for a python script?
Thanks
NB: if you can’t put spaces, replace them with special chars to be replaced by spaces.
July 5, 2008 at 2:50 am
Line 24 has two open parenthesis and four close parenthesis
wkey.append(ord(key[i*2]) 8)) ^ byteXorVal) & 0xFF
July 8, 2008 at 12:21 am
I ran across a problem with the ereader2html script and footnotes. As mentioned previously, the footnotes are not being decrypted with the rest of the book. I think this might have something to do with PML. From what I gather, the text of the footnotes are at the end of the pml file, but are ignored as part of the actual book text. I’m guessing that the self.num_text_pages (line 290) is only returning the information up to the first footnote. So, Darkreverser, any chance you’d take a look at this? (please?)
Thanks.
July 8, 2008 at 9:46 pm
July 8, 2008 at 9:47 pm
July 11, 2008 at 8:57 pm
Any of the mobi files at: http://lds.org/handheld/newarchive/0,18495,344-81-2,00.html
e.g. http://lds.org/handheld/pdafiles/pocketpc/famproc.zip
fail to convert, however the book has NO drm and works fine on any mobi reader.
The huff decode says: Error: The book is encrypted. Run mobidedrm first
But mobidedrm can’t easily be run as the book is not encrypted as far as I know. If I provide the PID of my ebook reader I get: Error: no key found. maybe the PID is incorrect
My only guess is thta maybe it is drm’d to a generic key that all readers have…?? Any clues on this?
August 5, 2008 at 8:52 am
The ‘0.03′ version given above references my first set of patches, but doesn’t fix all problems. Since then I’ve got more used to Python and found out a lot more about the Mobipocket format.
I now recommend some different patches for the 0.02 MobiDeDRM code. Detailed instructions on how to apply the patches can be found at
http://www.mobileread.com/forums/showpost.php?p=222142&postcount=125
HTH.
Paul
August 10, 2008 at 5:39 pm
Thanks for eReader2HTML!
I buy all my books legally, but cannot read them on my expensive BeBook without your help.
Still struggling to get mobi2HTML to work – with out an Kindle PID.
August 15, 2008 at 10:32 am
hmmm, this code does not work with copy/paste
test
must check8)it
test
September 4, 2008 at 1:03 am
Ah, thanks for the files! Sorry, this is a really basic python question since I am unfamiliar with it, but how do I get it to run the script?
September 5, 2008 at 7:56 am
Came in just to say a big THANK YOU to Dark Reverser for his eReader2html.py script. I had a few dozens of eBooks I bought over the years for my Palm and now I can read them on my N800, or wherever I want. Thanks again!
PS: I’ve shared my copy of the above scripts in the Donkey P2P network, search for “4DeDRMfiles.zip”.
May the good code live forever
September 17, 2008 at 4:17 pm
Hello
Please can you update the links ?
What is the latest version of mobidedrm.py ? and where can I find it ?
Thanks
September 24, 2008 at 5:30 pm
Thanks so much DarkReverser! I can now read my legally bought eReader eBooks on my Sony PRS-505!
I have re-uploaded the scripts to pastebin, as I had a hell of a time trying to find them.
eReader2html.py 0.03 : http://pastebin.com/f140eea7f
mobihuff.py 0.03 : http://pastebin.com/f35777523
mobidedrm.py 0.02 : http://pastebin.com/f2a681132
September 24, 2008 at 9:14 pm
Hi Dark Reverser,
Excellent work, judging by all the posts.
Unfortunately I had trouble with a book I bought and tried to deDRM to subsequently convert into pdf to read on my new Archos.
I got error messages with all three versions of mobiddrm, v0.01, v0.02 and the patch 0.04:
MobiDeDrm v0.01. Copyright (c) 2008 The Dark Reverser
Traceback (most recent call last):
File “MobiDeDRM001.py”, line 148, in ?
file(outfile, ‘wb’).write(DrmStripper(data_file, pid).getResult())
File “MobiDeDRM001.py”, line 117, in __init__
found_key = self.parseDRM(sect[drm_ptr:drm_ptr+drm_size], drm_count, pid)
File “MobiDeDRM001.py”, line 75, in parseDRM
pid = pid.ljust(16,”)
TypeError: ljust() takes exactly 1 argument (2 given)
MobiDeDrm v0.02. Copyright (c) 2008 The Dark Reverser
Traceback (most recent call last):
File “MobiDeDRM002.py”, line 176, in ?
file(outfile, ‘wb’).write(DrmStripper(data_file, pid).getResult())
File “MobiDeDRM002.py”, line 144, in __init__
found_key = self.parseDRM(sect[drm_ptr:drm_ptr+drm_size], drm_count, pid)
File “MobiDeDRM002.py”, line 101, in parseDRM
pid = pid.ljust(16,”)
TypeError: ljust() takes exactly 1 argument (2 given)
MobiDeDrm v0.04. Copyright (c) 2008 The Dark Reverser
Traceback (most recent call last):
File “MobiDeDRM004.py”, line 181, in ?
file(outfile, ‘wb’).write(DrmStripper(data_file, pid).getResult())
File “MobiDeDRM004.py”, line 149, in __init__
found_key = self.parseDRM(sect[drm_ptr:drm_ptr+drm_size], drm_count, pid)
File “MobiDeDRM004.py”, line 103, in parseDRM
pid = pid.ljust(16,”)
TypeError: ljust() takes exactly 1 argument (2 given)
Thanks again,
Sassie
September 24, 2008 at 9:18 pm
Oh, I forgot to mention, I’m using preinstalled Python 2.3.4 on Linux at the command line.
September 29, 2008 at 3:48 pm
I downloaded a fresh copy of Mobireader onto my PC and just downloaded a mobi book via booksonboard, when I use the PID fro Mobireader I get the following error
python mobidedrm1.py deaduntildark4.prc deaduntildark.mobi AAAA11A$1A
MobiDeDrm v0.01. Copyright (c) 2008 The Dark Reverser
Error: invalid PID checksum
(key changed to A for letter 1 for number)
I tried using v1 and v2 of mobidedrm and got the same result… did Mobi update keys?
September 30, 2008 at 10:57 am
Kier, you have to escape the $ sign. Like this:
python mobidedrm1.py deaduntildark4.prc deaduntildark.mobi AAAA11A\$1A
That should do the trick.
September 30, 2008 at 6:33 pm
And I’m a python programmer, I should have seen that, doh! worked like a charm!
October 1, 2008 at 8:45 pm
I would like to use ereader2html to convert a book that I myself compiled back into HTML. The book is unencrypted. Is there something I should use for the “name” and “credit card number” values to reflect this? It doesn’t seem to work if I just leave them blank.
October 2, 2008 at 10:42 pm
You have to have Pyhton 2.5 installed and the code posted here is useless as Pythin requires indents. 0.03 will not work properly as it has a bug in the updated code. 0.04 that I have has fixed the bug and works. But, you’ll have to have a proper 0.02 and then go to MobileRead and find the updated code for 0.02 to convert that into 0.04.
October 4, 2008 at 1:38 am
Many thanks Dark Reverser, JS Wolf and Paul Durrant.
Since there are so many posts, I just wanted to be explicitly clear how to make version 0.04:
you take the original version of mobidedrm 0.02 (eg from demonoid) and then simply apply the latest patch instructions referred to above by Paul Durrant on August 5 (and shown at mobileread) and voila you have version 0.04
Otherwise, some (including me) may get confused in thinking that you had to apply the patch to an updated 0.02 rather than the original 0.02
October 17, 2008 at 10:34 am
Help! If I try the following:-
python mobidedrm1.py name_of_file.prc name_of_file.mobi P3DA11A\$1A
The checksumPID function returns a PID but with the last two characters after the “$” different from the orginal. Therefore, the script fails with
“invalid PID checksum”.
October 29, 2008 at 7:20 pm
I got an error with mobidedrm 0.02 – I m decrypted 20 books or so but 3 or so books came back with the same error:
C:\Books>mobidedrm “A Forest of Stars.azw” “A Forest of Stars.prc” 3ASBJCF*KF
MobiDeDrm v0.02. Copyright (c) 2008 The Dark Reverser
Decrypting. Please wait…
Traceback (most recent call last):
File “C:\Python25\Tools\scripts\mobidedrm.py”, line 176, in
file(outfile, ‘wb’).write(DrmStripper(data_file, pid).getResult())
File “C:\Python25\Tools\scripts\mobidedrm.py”, line 159, in __init__
extra_size = getSizeOfTrailingDataEntries(data, len(data), extra_data_flags)
File “C:\Python25\Tools\scripts\mobidedrm.py”, line 74, in getSizeOfTrailingDataEntries
num += getSizeOfTrailingDataEntry(ptr, size – num)
File “C:\Python25\Tools\scripts\mobidedrm.py”, line 64, in getSizeOfTrailingDataEntry
v = ord(ptr[size-1])
IndexError: string index out of range
November 1, 2008 at 8:20 pm
I’ve used the ereader2html script successfully on nearly 300 books. However, I’ve got a couple of dictionaries that I’ve had no luck with.
Back in February, Kurt commented:
“Thanks for the eReader2Html script. It even works with dictionaries, simply they have different id: PDctPPrs.”
Can anyone help me figure out how to apply that info to make it work for me?
I also found this recommendation elsewhere:
With the ereader2html script it’s possible to convert a secure eReader pdb file into plain html. The original version doesn’t work with dictionary files but you can open the script in an editor, look for the “Invalid file format” error and change the raise into a print:
print ValueError(’Invalid file format’)
I tried that with both dictionaries, but to no avail. I’d really appreciate any help anyone can provide.
Many thanks.
November 1, 2008 at 8:38 pm
Regarding my previous comment, I was finally able to de-code both of my dictionaries using ereader2html. I had to change the script as noted (change raise ValueError to read print ValueError), and I had to use the original downloaded versions of the books, not copies. In my experience, Mobipocket books had required use of originals for de-coding, but not ereader.
Thanks to all for sharing your info and knowledge.
November 20, 2008 at 1:08 am
I bought a book from mobipocket that I want to get into my Sony 505.
I have tried to use the python script and have gotten close, but no go. I have been using MobiDeDRM.py with the command python MobiDeDRM.py body.prc body.mobi device_pid and get an error stating Error: invalid PID checksum.
Likewise I have tried to run Removes protection from Mobipocket books Usage: mobidedrm infile.mobi outfile.mobi PID. I want to kept the file a PRC file so that I can convert it to the Sony format in the Calibre software. I would be most helpful, by book has been taken hostage.
Thank you,
December 19, 2008 at 6:36 am
Latest MobiDeDRM.py v0.05 posted here:
http://pastebin.com/m57062830
Changes by Paul Durrant:
http://www.mobileread.com/forums/showthread.php?t=34190
December 21, 2008 at 5:27 am
Where do we get the PID from?
January 10, 2009 at 2:39 pm
This is a great script, thanks very much darkreverser. I have successfully stripped many files. However I have one that it doesn’t work with; I get a message saying “Invalid file format”. This file is readable using Mobipocket software, and Calibre recognises it as a valid mobi file. Any ideas on how I can investigate further to solve the problem?
January 10, 2009 at 9:39 pm
I am using ereader2html and cannot get it to work. I don’t know if I am entering the command wrong or what. I am using version .03 linked above in Bob’s post.
Thanks so much for any help you can give me.
John
January 10, 2009 at 10:35 pm
OK, I finally figured it out and it works great. Thanks so much for this great tool.
John
January 13, 2009 at 6:36 pm
Hi to all – have used the ereader2html script to convert a good number of Ereader files that I purchased legitimately. However, having left it for a few weeks when I try to run the script I am getting the Python error message that “ereader2html.py returned exit code 0″. Clearly I’m doing something slightly different from before and any hints or suggestions would be very welcome
January 13, 2009 at 6:42 pm
Worked it out muself. For other users who may come this way I was putting ereader2html in the arguments box – you only need the book name etc
January 19, 2009 at 9:57 pm
Hi,
I could use some help. While running the MobiDeDRM script I get the following error:
data_file = file (infile, ‘rb’).read()
NameError: name ‘file’ is not defined
Do you have any suggestions as to what might be going wrong? I’ve tried versions 1, 2, and 3 on several ebooks. All with the same result.
January 22, 2009 at 2:11 am
Went to get latest version and it is not there could someone help? need 0.05
January 22, 2009 at 4:54 am
Sorry I have absolutely no idea about computer stuff appart from clicking buttons in windows programs. Could someone give me an idiot’s guide for getting from the ereader2html/txt file that I download to the python script file that I run? I have Activepython installed, but other than pasting the text in to the python interactive shell (which doesn’t work) I have no idea! I have found instructions on running the ereader2html.py file from the command window, but how to get to the ereader2html.py is a puzzler!
Thanks, Jodie.
January 22, 2009 at 1:43 pm
OK I have the latest version I think, but here is my issue. I convert to a DRM free ebook and I have no images within the ebook. Am I doing something wrong?
January 29, 2009 at 6:14 am
MobideDRM.py 0.05 — http://pastebin.com/f31e88df6
January 29, 2009 at 6:15 am
MobiDeDRM.py 0.05 — http://pastebin.com/f31e88df6
This is the latest version.
February 2, 2009 at 2:52 am
This is so great. Now if only there were a version that worked with Topaz files so I could get rid of the ridiculous fonts Amazon forces on Kindle users with all those Kindle books they only sell in Topaz format …
Come on, Darkreverser, we have faith in you!
February 3, 2009 at 10:02 am
Been using ereader2html successfully for ages. Have now just bought my first non-DRM ereader book (unintentional) and came up with following message:
Error: incorrect eReader version 10 (error 1)
Help…
February 6, 2009 at 6:02 am
i ran 05 version in python 3.0 (should we only use 2.5?)
C:\python30> python mobidedrm05.py file.prc new.prc KKKKKKKK\$KK
i get this error
file “mobededrm05.py”, line 22
print “Bad key length!”
^
SyntaxError: invalid syntax
what am i doing wrong?
i got this same error for versions 1 and 2.
thanks
February 6, 2009 at 2:51 pm
leave out the \
February 7, 2009 at 8:47 pm
FOSSIL: I’ve got the same problem. It just stopped working today. If I comment out the (error 1), I get an (error 2) message.
February 7, 2009 at 10:10 pm
Update: ereader2html.py 0.03 is still working. However, by some act of major irony, UN-encrypted eReader books are providing an endless string of (error 1) wrong version messages.
Any suggestions?
February 7, 2009 at 10:55 pm
If the eReader file is not DRM laden, then eReader2HTML.py will not work to convert the eReader file to HTML.
February 8, 2009 at 6:58 am
Turned out not to be a problem. The old “pdbshred” program generates a WTF? message, but turns it into entirely readable PML. All I need to do now is turn the PML into HTML and I’m gone.
Thank you.
February 14, 2009 at 8:53 am
I’m using “MobiDeDrm v0.05″ and can’t get it to work. I don’t have much experience with this. I’m trying to get a file which opens on my Windows Mobile phone in Mobipocket to run on my XO/OLPC running FBReader. It says it needs a PID so I’m giving it the one from my phone and getting the error below. Am I doing something wrong? It looks like it should be rather straightforward…
C:\books>mobi.py Lion.prc Lion2.prc **********
File “C:\books\mobi.py”, line 22
print “Bad key length!”
^
SyntaxError: invalid syntax
Also, I just noticed, this error comes up because of a check for len(key)!=16. The key I am giving it is only 10 characters long because that’s what MobipocketReader gives me. Is that the wrong one?
February 14, 2009 at 9:13 am
Just wanted to say how great your scripts are, very much appreciated. Let’s me read my books on whatever device I want, instead of being stuck. Thanks.
February 16, 2009 at 2:12 am
Thank you for your wonderful scripts. I use them to convert books I buy so I can read them on my kindle. Without your scripts, I would buy fewer books – I would be restricted to one seller.
February 16, 2009 at 12:18 pm
10 is the correct length for a Mobipocket PID and the copy of 0.05 that I posted works fine with a 10 character PID. Maybe try downloading it again?
February 19, 2009 at 3:41 am
Looks like Adobe’s ePub DRM may have been broken
http://i-u2665-cabbages.blogspot.com/2009/02/circumventing-adobe-adept-drm-for-epub.html
February 19, 2009 at 8:46 am
Will encryption type 1 documents ever work? I just found out I deleted a DRM’d file, which was meant for my PDA. (I still got the one that works on my previous PDA…). I can’t DeDRM it, as it is a encryption type 1…
February 28, 2009 at 3:16 am
Phil, your problem could be that you might be using Python 3.x. MobiDeDRM.py does not work with Python 3.x. It only works with Python 2.5.x or 2.6.x.
March 12, 2009 at 5:22 pm
Okay. The perennial question: How does one use ereader2html for ereader, PDB books that do NOT have DRM? I have a bunch of free, non-DRM ereader files that I would like to use with FBReader, but FBReader can’t read ereader files. It will work with HTML files, mobi files, epub files, and some others, so I need to convert them somehow. There IS a pdb2txt utility, but that doesn’t always work for me, and when it does, it doesn’t retain the formatting.
So, IS there a way to get ereader2html to work with non-DRM ereader files? Any other ideas?
Thanks!
March 15, 2009 at 5:00 pm
I have some VERY OLD (peanut press) files that I would like to read again. They seem to be too old for ereader2html.py (version 259 and earlier). Any ideas?
March 17, 2009 at 7:58 pm
Looking for the tool mobidedrm and clicking on the link above for your latest script seems to bring up a broken/deleted page?
March 17, 2009 at 8:51 pm
Look at my post on January 29, 2009.
March 18, 2009 at 4:20 pm
I have created a program that incorporated the mobidedrm (v1.0) and the kindlefix scripts into a gui and made the whole thing a windows exe. No python needed. Right now it only converts .azw drm books and does the mobi conversion so that a mobi drm can be read on the kindle. If someone is interested in helping me test it further I’d need a SN from a kindle and a .azw drm’d book that was bought with it. I also need a .mobi drm book to test the other function.
March 18, 2009 at 4:22 pm
Sure, I can provide everything you need. E-mail me to tell me how to get them to you.
March 18, 2009 at 4:40 pm
chorpler,
Send mail to the temp address msg4bill-temp@yahoo.com and we’ll talk
March 20, 2009 at 2:00 am
Thanks so much for providing these scripts.
Just wondering if you’ve made any progress on the footnote issue that others have mentioned above.
Great work, and thanks again.
March 29, 2009 at 6:38 pm
i am a total idiot when it comes to computer stuff. so a little patience, please.. so i can’t get python to run mobidedrm v.01. i typed in the script and then what? do you type something in to get it to run? how exactly do i get this to work? i just want to remove the damned DRM’s on my mobipocket books to convert it onto my sony reader. thanks
March 29, 2009 at 6:41 pm
oh and where would i actually go to start removing DRM’s from the books?
March 30, 2009 at 5:47 am
I got .05 pieced together, worked like a charm. Seems like the pieces/parts are all over the place to put this together, so I put together an archive that includes all the related files in one place:
http://rapidshare.com/files/215202363/Mobipocket_DRM_Tools.zip
This puts together Igor’s files including kindlepid.py, Darkreverser’s MobiDeDRM.py .05, and the Applescript GUI from mobileread.com forums (non-handicapped, MobideDRM already installed). Also updated the various readme files to help get the newbie up and running more quickly.
March 31, 2009 at 4:00 pm
I’ll be the first to admit I’m new to this and not at all familiar wiht programming. I do have rather a lot of ereader books that I’d like to use on a Kindle. I’ve installed Python 3.0 for windows. when I run python I get a terminal window. I then type ereader2html.py c:\destination folder\ ‘My Name’ ‘number’. I get “invalid sytax” every time. I don’t know what that means. I’ve tried changing the syntax somewhat or added python to the command line, but still no luck. I wonder what I’m doing wrong. I’ve tried moving all the files to the same directory, etc. Can anyone give me a tip? It all seems so simple when reading about it. Thanks for any help you can give me.
March 31, 2009 at 4:03 pm
Sorry, I do also include the name of the book as well with the pdb ending.
April 3, 2009 at 6:30 pm
Thanks for the version 5 of drm. I incorporated it into a VisualBasic program that I created. If anyone would like to try and run it send email to nodrm.50.shootmel8ter@spamgourmet.com. You can use this on a windows system without loading python.
April 7, 2009 at 9:48 pm
mm… funny.
April 8, 2009 at 2:13 am
Jim, I know absolutely nothing and can’t get the script to work, either. But I do understand that you should try to use Python 2.5 or 2.6, not 3.0. Hope this helps. (And if it does, and you get it to work, maybe you can help me!!)
April 10, 2009 at 6:12 am
Thank you for a lot of PRC files I have converted.
But I cannot use the Mobiderm code to decrypt the following .PRC file.
http://www.mediafire.com/?e3hbragmmmk
April 11, 2009 at 9:21 am
Jane, can you tell me what’s your exact problem ?
Contact me on this email : xr17hkup7ldli3n@jetable.com ,and I will help you
April 13, 2009 at 7:08 am
If you check out this thread over at Mobileread:
http://www.mobileread.com/forums/showthread.php?t=34322
You’ll see that Calibre now supports the ability to use something like MobideDRM as a plugin. Can anyone post some instructions here to pluginize MobiDeDRM, or better yet just post a link to a plugin?
April 14, 2009 at 4:00 am
Dang, the last 7 hours and I still cannot be sure how this works! Can anyone email me at pilotrite AT aol to give me some guidelines? This is so frustrating. I just want to be sure I can read my books I’m paying for if the kindle 2 ever goes belly up. And just in case text to speech disappears, id like to be able to convert and still use. HELP!
I finally gave up tonight after getting the following error… Error: invalid encryption type: 0. UGGGHHHHH! I’m so frustrated! HELP! (Oh, and is there a fix for topaz or azw1 files tooo yet? Of course I can’t get the azw one to work yet!)
April 20, 2009 at 2:50 am
Does anyone know where to get mobidedrm05? Thanks!
April 20, 2009 at 2:53 am
Nevermind I found it a few posts above mine
I used mobidedrm04 all the time before but now I get a bad key length error and it doesn’t run… 02, 04, and 05. Help? Thanks!
April 20, 2009 at 3:02 am
Figured it out, doesn’t work with Python 3, so I downloaded Python 2.6. All systems go
April 30, 2009 at 4:50 pm
Wow, meriweather. you da person. thanks.
May 14, 2009 at 4:10 am
Meriwether that is one heck of a AppleScript. Thank you very, very, very much. Without the help of you and the others I would have returned my Kindle a few days after buying it and understanding the DRM issue. I have purchased now about 30 books from Amazon and Fictionwise. But I would have been stupid to have bought any if DRM could not be stripped. As an early adopter of technology I am pretty darn sure my next ereader will show up in the next year or eighteen months and will not be Kindle (might be but I doubt it). Amazon and the other companies and sites are being silly to try to block this. If I wanted to pirate there are twenty torrent sites I could use. But I’m not a pirate. But neither do I want to be a patsy and have to buy the same books over and over and over again as years go by.
May 16, 2009 at 7:34 am
Thanks, but not my Applescript – thank pdurrant from the mobileread forums. I just packaged it all together.
May 17, 2009 at 12:34 pm
Hello,
I’m a PRS-505 owner. I bought yesterday 3 books at mobipocket store, thinking that it’would be easy to convert them to lrf with Calibre. I didn’t realize that it would come with DRM.
Then looking on the web for a solution I finally arrived here. I installed pyton and dowloaded the 4DeDRMfiles . My problem is that I’m a complete newbie in this kind of things and after spending a long while trying differents ways, I don’t understand at all how it works. Does-il exist somewhere a tutorial explaining step by step how to proceed?
Thank you very much & and have a nice day!
May 17, 2009 at 7:39 pm
Hello again,
after many investigations, I finally found “the way”.
Thank you again
May 23, 2009 at 6:33 pm
I have recently purchased “Foundation (Isaac Asimov)” in the Kindle edition, only to discover the TTS is disabled. I use TTS a lot in the car, as I have a long commute. Copyright law being on my side, I can use the mobi2mobi program with the “–delexthtype 404″ option to remove the disable TTS flag. When I put this file BACK on the Kindle (still containing the DRM) it works fine, and even Whispersync still works between my K2 and my iPhone.
However, I like to use mobidedrm on ALL my AZW purchases and convert them to plain mobi for back up, in the event that the whole Kindle thing goes belly up. In effect, I refuse to acknowledge that I’m only LEASING an eBook for Amazon. I paid for it, I think it’s fair use to strip it and convert it to something I can read on my Sony or Stanza or whatever.
Here is the problem. Mobidedrm creates a corrupt file from Foundation.AZW…it crashes Mobipocket reader, mobi2oeb (Calibre), etc. It does this when I use mobidedrm before the TTS removal, or after…makes no difference. It’s almost as if Amazon has found a way to defeat mobidedrm.
Anyone have a clue about this?
May 23, 2009 at 6:41 pm
Answering my own post here, I found an OLDER version of mobidedrm (version 1, actually compiled into an exe mobiddrm01.exe) and it removed the drm on Foundation.AZW just fine.
June 4, 2009 at 1:02 pm
Have you been able to create a mobidedrm script for AZW1/Topaz files. If not do you anticipate their being one any time soon.
June 5, 2009 at 9:38 am
Hi,
I recently came across some mobi e-books that have TEXtREAd as file format. v.0.5 isn’t able to remove the DRM. I tried editing the code to let it accept TEXtREAd files and that worked, but it still can’t handle the file. I read elsewhere that it was once possible to remove DRM from this kind of files with an edited tool from the iRex devices, but it’s impossible to get that now. Do you know of a way to remove the DRM of TEXtREAd mobi files? Perhaps it’s very easy to write code for it, but my python skills aren’t really great
Thanks for any help!
Sander
June 23, 2009 at 10:17 pm
I purchased a *.mobi e-book recently, that cannot be cleaned from the DRM infestation with any available mobidedrm version; mobi2mobi sees the following metadata:
./mobi2mobi my-precious-ebook.prc
Database Name: XXXXXXXX
Version: 0
Type: BOOK
Creator: MOBI
Seed: 7307789
Resdb:
AppInfoDirty:
ctime: -1968466898 – Fri Aug 16 19:13:01 1907
mtime: -881913230 – Tue Jan 20 17:06:10 1942
baktime: -2082844800 – Thu Dec 31 23:34:39 1903
—————————————————
FIRST IMG Record Id: 7307783
—————————————————
Image record index: 519 (90 x 120)
START IMAGE INDEX: 519
COVER IMAGE INDEX: 519
PDHEADER Version: 2
PDHEADER Length: 1059157
PDHEADER NRecords: 518
PDHEADER Recsize: 2048
PDHEADER Unknown: 190892
MOBIHEADER ciflg: 65535
MOBIHEADER ciptr: 65535
MOBIHEADER doctype: MOBI
MOBIHEADER length: 228
MOBIHEADER booktype: 2 – BOOK
MOBIHEADER codep: 1252
MOBIHEADER uniqid: 2853551425
MOBIHEADER ver: 4
MOBIHEADER exthflg: 64
MOBIHEADER language: 9 – 9 – 0 – ENGLISH –
MOBIHEADER xtradata: 27651 0×3237363531
COVEROFFSET: 4294967295
THUMBOFFSET: 4294967295
EXTH doctype: EXTH
EXTH length: 84
EXTH n_items: 5
EXTH item: 201 – CoverOffset – 4 – 0xffffffff
EXTH item: 202 – ThumbOffset – 4 – 0xffffffff
EXTH item: 203 – hasFakeCover – 4 – 0×0000
EXTH item: 2 – drm_commerce_id – 14 – EBOOKMALL_0306
EXTH item: 3 – drm_ebookbase_book_id – 5 – 18668
LONGTITLE: XXXXXXXX
LASTID: 7307786
Were there any changes to mobidedrm since the v0.05 release?
June 30, 2009 at 3:49 am
Hint about ereader2html.py:
Footnotes each have there own page. The first page is at byte 44 of the header (like the first image page is at 24). Number of footprints / sidebar items is at byte 46.
Encoding is same as text.
From this it should be easy for any of you python coders to add this feature.
This only works for type 272 files, of course.
July 3, 2009 at 10:09 pm
Trying to find ereader2html.py but the link in the main blog page no longer works. Having no luck with Google. Anyone have a current link to it?
July 7, 2009 at 7:25 pm
Thanks for this site and this scripts which had given a second life to my ebooks bought from Mobipocket.
I use a MacBookPro and a iPhone with the Stanza reader.
Today, I have bought 5 ebooks from Mobipocket for the first time since two years.
The applescript works fin with 2 of them.
But I have a problem with the three others, I think recently published on Mobipocket site.
The applescript seems work fine, but when I try to open the unlocked book with Stanza, I get this message :
Could not load book
java.lang.ArrayIndexOutOfBoundsException : 2182
It’s the same with the three others scripts Mobidedrm 1, 2 and 5, on terminal windows on Mac.
What’s the matter ?
Is it a new defense or new format of ebooks from Mobipocket ?
Thanks for your response.
July 15, 2009 at 2:54 pm
Oh Dark Reverser — Please, please please advise as to AZW1/TPZ files. Anything available or in the works?
July 20, 2009 at 7:05 am
@Babar, Stanza is pretty buggy with lots of Mobipocket titles (nothing to do with MobideDRM), try using Calibre with the decrypted files instead. If you’re using Stanza on the iphone you can use Calibre’s content server to send books to the device.
For Mac users also using their iphone to purchase kindle title from Amazon, there is s new tool out there for you (not mine):
http://github.com/tradewinds/TradeWinds/tree/master
This automatically extracts the books from your iphone/ipod backup directory, simultaneously decrypting them.
July 21, 2009 at 1:31 am
Found and have been using ereader2html and the ER application OK on my PC. Stripped a bunch of books I had purcahsed (I do pay — once — for books). But today Barnes and Noble opened their new ebook store. The ER application fails on the books I have from them. Does anyone know how it could be made to work or why it is failing? PLEASE reply, thanks!
July 21, 2009 at 1:41 am
Further Tradewinds referenced above seems to have been taken down already. I am batting zero taody. Damn DRM anyway. Is this Tradewinds thing available anywheres else? Couldn’t find any place else, also doesn’t help that “tradewinds” means so many things.
July 21, 2009 at 1:59 am
Oooops. I apologize — I was making the same, stupid typographical error every time I was entering a book title in the script. The good news is that Ereader2html DOES work on the new Barnes and Noble ebooks. I feel like a bit of an idiot but would rather correct my error then post misinformation here!
July 22, 2009 at 5:37 am
The actual download link for Tradewind’s application is:
http://cloud.github.com/downloads/tradewinds/TradeWinds/TradeWinds.dmg
July 28, 2009 at 11:18 am
@ meriwether
Thanks for your answer
I tried Calibre, but it can’t open these books entirely. It fall 3/4 of the book.
Now, I have the same problem with the quadrilogy of Twilight, bought by Nulmerilog, one of the 4 book is unreadable by Stanza, after the decruyption. The three others are OK ?
Thanks for the link for TradeWinds, i’ll try it.
August 3, 2009 at 3:40 am
Just thought I’d drop a line to those interested, I made an easy, GUI, batch-capable wrapper for the mobidedrm python scripts. Nothing fancy, but it’s got its uses. http://www.mediafire.com/file/dlmkdzzzm2n/eBookUtility0.2.zip This version has an exe wrapper and is pretty windows specific but if anyone wants it for a *nix platform I can put out a platform independent jar. I know some other work has been done around this that’s independent of the python scripts, this is nothing like that, mostly just a convenience / user friendliness issue. I created it for family members who own kindles and aren’t quite so tech savvy, and I like to use it for batch stripping. This is pretty much my first time releasing it to an audience beyond immediate family, so feedback is welcome.
August 6, 2009 at 12:43 am
I’ve created a patch for eReader2html003.py to enable decryption of “version 259″ ebooks.
Only had one example to work with so can make no guarantees, but it’s worth a try right?
Major props to DarkReverser for laying the groundwork and to the author of palmdrm.txt for providing a vital clue.
eReader2html003-to-003b.patch:
http://pastebin.com/f147cc228
remember, authors have to eat too. so if they’re still alive, buy their ebooks…
..AND THEN SET THEM FREE!!!!
August 7, 2009 at 9:17 pm
I see that there are several people reporting Mobipocket files that won’t decode properly, or that decode with earlier versions but not 0.05.
I thought that with 0.05 I had got the decoding working on all Mobipocket DRMed files.
I would be interested in seeing examples of any file that won’t decode with 0.05.
pdurrant (not the Dark Reverser)
August 9, 2009 at 9:11 pm
Hey, revx, thanks for the kinder, friendlier code! Do you have tips for how to change the default paths the .exe is checking? My python install isn’t on C, etc. Thanks!
August 10, 2009 at 2:36 am
@Paul Durrant et aliis
I’m grateful for everyone’s heroic efforts here. Paul, I today bought a dictionary from Mobipocket that won’t pass through 0.05. (My first purchase from Mobipocket. The program worked fine with an .azw file created from a .prc from another vendor.) I’d be happy to send it to you, but it’s a big file. Tell me if you want it.
The error is either “no key found. maybe the PID is incorrect,” when I pass my kindlepid.py-generated PID to the program or “invalid PID checksum,” if I pass it the PID without the “*” or if I substitute a “$” for the asterisk.
As previously reported by many, Mobipocket stripped the asterisk from a Kindle PID when I registered it for the download (as I set up my account). Presumably, though, it embedded some kind of key in the book as it DRMed it. Can we assume that key is based on either that 9-character PID (original minus the asterisk) or on a 10-character PID it created from that original PID plus some other character substituting for the asterisk? Can’t someone figure out what manipulation they perform on the mutilated Kindle PID?
I’m working on a Mac, so the desktop PID route is not open to me.
August 11, 2009 at 7:00 pm
@egret
Yes, good point on the Python check, I’ve simply removed that check for now, I’ll reimplement it later when I’ve got more time. In the mean time, here’s an updated version and it just assumes you’ll treat it nicely and have a working version of python installed. (i.e. there’s an association so if you type python at command line it’ll open python)
http://www.mediafire.com/file/m4mhin2wmg2/eBookUtility0.3.zip
August 12, 2009 at 12:10 am
Oh, thank you! I had the association working and was head desking pretty hard after that.
Where the heck does it expect to find the PID? I don’t have a kindle, just a mobi PID…
August 12, 2009 at 12:24 am
Let me report a further odd occurrence in case it provides a clue to someone who understands the guts of these programs better than I do.
(Recall: I’m faced with the now banal–but not trivial–problem of MobiDeDrm 0.05 not running to completion because of Mobipocket stripping the “*” out of kindlePID-generated PIDs at purchase.)
While waiting for some sort of encouraging reply, I was flailing around trying to do something. I was examining the not very imaginative hypothesis that Mobipocket was substituting another character for the asterisk when generating the key for the .prc file. So I was substituting characters myself, hoping I’d get lucky and one would work. No such luck, of course.
But I did find two other characters which passed the checksum test, though they went on to fail whichever next test returns the error “No key found. Maybe PID is incorrect.” The asterisk–ASCII 2A–passes the checksum. But so do ASCII 2F (forward slash) and ASCII 8D (unassigned). I can’t imagine how such different values can all pass the checksum. Maybe it’s just an arithmetic fluke. (For what it’s worth, 2F is one-third of 8D, but nether has any evident arithmetic relation to 2A.) But maybe, since it seems anomalous, it’s a clue to someone. So I pass it along. We live in hope.
August 12, 2009 at 12:55 am
@Chris
Shouldn’t be too hard, here’s all you need to do, I believe.
Create a file in the ebookutility folder called ebook.props (plain text file) and put the following in it:
PIN=
serial=
August 12, 2009 at 12:57 am
Woops, my last comment got a clipped a little, to clarify:
PIN=YourPID (i.e. PIN=8KL*KDJF)
serial=RandomJunk (You don’t have a kindle, so don’t worry about it just give it something bogus, 1234 for instance.)
August 12, 2009 at 1:00 am
Danke!
August 12, 2009 at 1:59 am
Hi Revx — I’ve been using the cripts on my Mac and also the Applescript utility without problems. Today I saw that you had a GUI for Windows and downloaded the ebookutility program hoping to get my Windows machines working on it also. However, the ebookutility program will not recognize my Kindle’s serial number. When I enter it the program gives an invalid serial number error and will not generate a PID for the DRM functionality to work. However, using the same serial number with kindlepid.py on my Mac generates a valid PID and on the Mac no problem stripping the DRM. Any ideas why that serial number is failing in ebook utility? Thanks!
August 12, 2009 at 2:10 am
@Neil
Not sure why it’s not working, but all it’s doing is making a call to kindlepid.py, so if the one on your mac works for your serial, pull it off and throw it in the ebookutility directory and see if it things works, if so, you should be in business. Also, if that still doesn’t work for some bizarre reason you can follow the directions I gave a couple posts above and just feed it your generated PID manually. Not pretty but it would work.
August 12, 2009 at 11:04 pm
FWIW, I used the unexpected arithmetic behavior regarding checksums that I mentioned in my last post to some advantage today. I went to another vendor for the DRMed dictionary I wanted. This vendor didn’t strip the asterisk from my Kindle PID. It simply wouldn’t accept it as a valid PID. (Nor would it accept it if I changed the asterisk to a dollar sign.) Because I was stuck, I used one of the other characters which passed the checksum test in place of the asterisk, and the vendor accepted it! When I then ran mobidedrm v. 0.05 on the resulting .prc file I downloaded from the vendor, it successfully stripped the drm from it. I have no idea whether this strategy would work across the board, nor whether it would have worked successfully with Mobipocket.
August 14, 2009 at 2:53 am
Can someone please repost the scripts as pastebin no longer has them. Use gist.github.com or something. thx.
August 17, 2009 at 12:06 am
Thanks for the awesome GUI revx. Makes life much easier. Now if we could only dedrm topaz files……
August 19, 2009 at 7:20 pm
To all who are interested, I release a new version of eBookUtility, the link is here
Change people care about: after hearing a few rumblings about it I allowed PRC files to be used as source files as well as AZW, it makes perfect sense to do so and probably should have been done in the first place, but I only had the kindle store on the brain when I first wrote it.
Things that might break: I reimplemented the python check in a much less idiotic way, if it gives you trouble this time around, however, I’ve included directions in the readme which explain how to bypass it.
August 20, 2009 at 10:01 am
I’m probably stupid but it is not working for me, I put my mobipocket reader PID and a serial (one junky because I don’t have Kindle) saved it as book.props.txt but it still asks Kindle serial number. What am I doing wrong
August 20, 2009 at 10:57 am
You have to save the file as “ebook.props”, read the ReadMe.txt file that comes in the zip, it gives you directions for that.
August 21, 2009 at 3:13 pm
Can’t find a copy of eReader2html.py v3 anwhere- no longer on pastebin
August 24, 2009 at 9:10 pm
Thanks for the help, tried it as suggested and works like a charm.
August 25, 2009 at 10:07 pm
I’ve just installed Mobipocket Reader on my PC (not having a portable reader of any sort yet), and purchased a .MOBI book (not Amazon). The MobiDeDRM python scripts fail to work on it. In looking at the script, it LOOKS like it’s using “ABCDEFGHIJKLMNOPQRSTUVWXYZ123456789″ as the legal characters in a PID, and the PID I get from the Mobipocket Reader program has a ‘$’ in it. Perhaps that’s why the scripts fail? Maybe Mobipocket has modified it’s latest versions to break the DeDRM scripts?
August 26, 2009 at 1:40 am
No, several of my PIDs have had dollar signs in them too, and they work fine for decryption. You might have to put the PID in quotes or single-quotes … are you using version 0.05 of mobidedrm.py?
August 26, 2009 at 5:12 am
Yes, I’ve tried versions .05, .02 and .01, tried with and without quotes. All give the same … err…. ummm….
shit.
I swear, I tried all three .py scripts, all three gave the same “PID checksum error” (or whatever). I tried it with quotes and without quotes, single quotes and double quotes, I tried rebooting Windows, I tried running the scripts on Vista and on XP, I tried it sitting down, standing up and standing on my hands. All gave the same error.
Then I tried it again just now, to get the exact error message… and it worked.
Sigh.
Never mind.
August 29, 2009 at 7:33 pm
I’m releasing a new version of eBookUtility with a serious bug fix which prevented the program from decrypting entire directories correctly. (Thanks to all who provided feed back) It also fixes a number of other minor bugs, such as correctly reporting when it can’t find python. The next version (0.5) will have more advanced threading and concurrency controls.
You can get the bug fixed version of (0.4a) at http://www.mediafire.com/file/mykmlzwz3gm/eBookUtility0.4a.zip
August 30, 2009 at 6:46 am
For those of u who successfully strip the DRM from your mobi file, can you please explain in steps by steps what to do? I have no idea how to do this and would greatly appreciated if someone can help me out. I probably need python and mobidedrm right? Where do I get those?
August 30, 2009 at 8:07 pm
The ereader2html003b.py code is on Pastebin here:
http://pastebin.com/f629041a4
Is anyone working on DeDRM for Amazon’s TOPAZ files?
They are a real pain in my butt…..
Cheers
August 30, 2009 at 8:21 pm
Superstitious:
First go your Mobireader, HELP|ABOUT and write down your mobi PID, It will be something like:
xxxxxxx$xx
Then put the mobi or prc file you want to decrypt in the same directoty as the dedrm script. I am assuming that you have python 2.6 installed in the c:\python26 directory.
Type the following:
c:\python26\python.exe mobidedrm05.py infile.mobi outfile.mobi xxxxxxx$xx
If all goes well, you will have an exact copy of the original (DRM’d) file with the name of outfile.mobi, but with no DRM.
Cheers
August 31, 2009 at 4:12 am
http://www.mediafire.com/file/mtwndjtzgzl/eBookUtility0.4b.zip Pushing out another major bug fix.
September 3, 2009 at 2:43 pm
Perfect. Thankyou. Precisely what I needed.
September 6, 2009 at 10:09 pm
Anyone know of a plugin for Calibre that makes use of these scripts?
September 7, 2009 at 12:35 pm
I’ve found one or two Mobipocket ebooks where 0.05 decrypts, but with some corruption in the text.
The fixes to make 0.05 into 0.06 are as follows:
Change the the print line (line 174) to read
print “MobiDeDrm v0.06. Copyright (c) 2008 The Dark Reverser”
Replace lines 76 to 82 with:
[tab]testflags = flags >> 1
[tab]while testflags:
[tab][tab]if testflags & 1:
[tab][tab][tab]num += getSizeOfTrailingDataEntry(ptr, size – num)
[tab][tab]testflags >>= 1
[tab]if flags & 1:
[tab][tab]num += (ord(ptr[size - num - 1]) & 0×3) + 1
[tab]return num
Add to the comments at the top:
# 0.06 – And that low bit does mean something after all
Note that [tab] should be replaced by actual tab characters. Python is sensitive to the amount of indentation on each line.
September 7, 2009 at 2:49 pm
Help. I ran out of dumb luck. I downloaded another drmed mobipocket from the vendor I used in my Aug. 12 (11:04 p.m.) report. But this time, when I ran mobidedrm 0.05 on it, I got the error message: “No key found. Maybe the PID is incorrect.” When I used the PID as I’d altered it to get the vendor to accept it–which involves substituting a forward slash for the asterisk–mobidedrm ran, but hung (as I discovered when I control-C’ed out of it) at line 185.
Also sinister: when I tried running kindlefix.py on it, that too failed, reporting back:
“Encryption: 2
“Mobi publication type: 2
“Mobi format version: 6
“PID doesn’t match this file”
That was the first time I’d seen any mention of a “Mobi format version 6.” Is there anything that can be done about this? Thanks for any help.
September 9, 2009 at 11:27 pm
Hey Paul, having a problem:
error: compiling ‘MobiDeDRM06.py’ failed
SyntaxError: invalid syntax (MobiDeDRM06.py, line 79)
I posted the code here, can you take a look?
http://pastebin.com/f6cbeb3f9
Cheers
September 9, 2009 at 11:30 pm
Sorry, line 80 not 79, made a typo.
Cheers
September 9, 2009 at 11:46 pm
Never mind, got it working. Some weird invisible characters in the code I pasted from your post. I retyped those lines by hand and now it compiles.
If anyone want’s it the full MobiDeDrm06 script is here:
http://pastebin.com/f6590ef1d
Cheers
September 10, 2009 at 7:30 am
There are two faults in the code brutusbum posted on 9/9 at 11:56:
Line 145 that reads
extra_data_flags, = struct.unpack(‘>L’, sect[0xF0:0xF4])
should read
extra_data_flags, = struct.unpack(‘>H’, sect[0xF2:0xF4])
and line 77 that reads
flags >>= 1
should be deleted – it shouldn’t be there at all.
Paul
September 10, 2009 at 10:16 pm
That’s the problem with posting just updated lines, the code is only as good as what you begin with. I wondered about line 77, but both versions I found of version 0.05 had extra lines near the beginning of the file. I will update and repost.
Version 0.0.6 Dedrm with the above corrections here:
http://pastebin.com/f29c0eef0
Cheers
September 13, 2009 at 6:52 pm
I tried the latest script and this is the result with the correct PID:
MobiDeDrm v0.06. Copyright (c) 2008 The Dark Reverser Error: no key found. maybe the PID is incorrect.
September 13, 2009 at 7:17 pm
@Drmed
It’s not a problem with the script. Unless you’ve previously had success with earlier versions?
If your PID comes from a Kindle, then some eBook sellers won’t accept the PID (with a $) as valid for Mobipocket books.
Use a different PID (e.g. from Windows Mobipocket Reader), re-download, and all should be OK.
September 13, 2009 at 8:13 pm
Thanks that worked
September 14, 2009 at 9:15 pm
Just wanted to say THANK YOU for this excellent program. Total python n00b but was able to figure it out with help from some forums. Looking forward to having my books available on Android, at least until ereader puts out an Android-compatible reader. Thanks again!
September 18, 2009 at 5:17 pm
After a few weeks of development, I’m releasing version 0.5 of eBookUtility, in both a Windows exe wrapped version and a platform independent jar.
Windows:
http://www.mediafire.com/file/kwiwzwmntgj/eBookUtility0.5-win.zip
Platform Independent:
http://www.mediafire.com/file/wjmr1j2zioh/eBookUtility0.5.zip
Changes:
eBookUtility 0.5 – 09/18/2009
—————-
- Complete revamp of the underlying threading system and external process call methods
- All selection method bugs appear to be squashed
- NEW: Thread monitoring tab with progress bar
- NEW: Only dedrm errors are output instead of all output
- NEW: 5 threads run concurrently to process selected book(s) (assuming you’ve selected 5+ books)
September 18, 2009 at 7:33 pm
Revx — Just wanted to thank you for version .5 of the Ebook Utility and to let you know that the .jar version works great on my Intel iMac. I wasn’t sure if it was MobideDRM version 6 or not included so replaced it just to be sure but am betting that was an unneeded step.
September 18, 2009 at 7:57 pm
@Thothamon
I hadn’t actually gotten around to patching my copy of MobiDeDRM to 6 yet, so that was indeed a needed step. I’ll get it patched for the next release.
September 19, 2009 at 4:40 pm
Using the ereader2html003b.py code my version 259 file was decrypted but I got the following warning:
eReader2html.py:11: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
import struct, binascii, zlib, os, sha, sys, os.path
I don’t know if this is important or not…
Thanks for making this work for version 259!
October 9, 2009 at 6:12 am
I have successfully used ereader2html003b.py, but the book I’m trying to convert is translated from Spanish, and therefore has a number of accented characters. Unfortunately, these aren’t coming through correctly. Many of them come through as Cyrillic characters.
Any suggestions?
October 9, 2009 at 6:16 pm
Thanks for the great programs.
I have used ereader2html v3 for my legit ebooks without any problem until this file. File is freebie from BN, have stripped other files from BN & FW with no problem. Using ereader2html with python25. Re-downloaded file in case orig was bad but get same message. File opens & reads fine in BN reader.
This is the error message I get with this file: (re-typed as I don’t know how to copy from command prompt)
File “e2hv3.py”, line 490, in
convertEreaderToHtml
File “e2hv3.py”, line 477, in convertEreaderToHtml
file.write
File “e2hv3.py”, line 417, in process
r = self.next
File “e2hv3.py”, line 349, in next
c = self.s[p+1]
IndexError: string index out of range
My python skills are just at the beginning level so I’m not quite sure where to start first. It’s a free book so if it’s corrupt somehow, I’m not too upset. Out of 100+ books, it’s the only file that I’ve not been able to convert.
Thanks for any help
October 22, 2009 at 3:10 am
I’ve downloaded the 0.03b version of the ereader2html python script, but I’m having some trouble getting it to work. When running the script I’m getting the error “incorrect ereader version 10 (error 1).” Unfortunately I don’t know what this means, nor what information to provide to help in the trouble shooting. I’ll be happy to answer any questions.
October 22, 2009 at 7:29 pm
Thanks for MobiDeDRM! It works great on my purchased books!
October 23, 2009 at 9:03 pm
I have two Amazon files (azw) that give me an error stating that it has encounter invalid encryption: type 0.
I am using Active Python 2.6, and both mobidedrm005 and mobidedrm006 with no differences.
I would be happy to provide these files (and the PID) to someone who wants to try to figure out what the heck is going on.
Many thanks,
October 23, 2009 at 9:09 pm
Encryption type 0 means no encryption. Not all eboks in the Kindle store have DRM applied to them.
October 26, 2009 at 4:50 am
Is there a script available that would just remove the drm from ereader .pdb books without converting to html?
October 26, 2009 at 5:50 pm
I’m releasing the newest version of the program. I’ve updated the ReadMe to include in detail instructions on how to solve the Python path issue, please read it if you’re having problems!
Windows:
http://www.mediafire.com/file/m0mmu4mfvzn/eBookUtility0.6-win.zip
Platform Independent:
http://www.mediafire.com/file/ymnn1zm2xmj/eBookUtility0.6.zip
Changes:
eBookUtility 0.6 – 10/26/2009
—————-
- Not much new in this release, I haven’t had any reports of bugs, and only one feature request.
- NEW: Added a checkbox to allow overwriting of the source files — many people didn’t like the ‘-stripped’ name addition, and it was easy enough to implement, so now you have it.
- NEW: The DeDRM scripts provided with the utility are now 0.06 so there’s no need to upgrade.
October 26, 2009 at 8:37 pm
Thanks to some prompt feedback I was able to catch a severe bug in the 0.6 release and pulled it immediately. I’ve reuploaded it and the new files can be found http://www.mediafire.com/eBookUtility here.
October 26, 2009 at 10:37 pm
Thanks for the update!
October 29, 2009 at 9:48 pm
Hi–anyway to do batch conversions with the mobi script?
P.s:
Note that for non-drm conversions, in response to a question, Amber Palm Converter, is a simple point and click solution for prcs and pdbs…
October 29, 2009 at 11:33 pm
Question–with the ereader script…it keeps telling me the sha module is deprecated, use hashlib instead.
Alas, I have no idea what this means, let alone how to fix it. Python 2.6 and 3.1 both installed…Any help would be appreciated.
October 30, 2009 at 1:17 am
A sincere thanks to revx for eBookUtility.
October 30, 2009 at 10:53 am
@Mitch
My utility is specifically designed for doing batch stripping with the MobiDeDRM scripts, check a couple posts up for the info.
As to your second question, it’s saying that a method used in the script is deprecated, meaning that newer functionality has been released in the Python language that replaces the functionality that the script currently uses. It’s not an error, it’s a warning. (i.e. when the script was written — perhaps I don’t know this for a fact — the sha module was probably current and what was available to coders and in later versions of Python hashlib was released)
October 30, 2009 at 11:40 am
Ok, thanks….is there a way to fix this? I have both Python 2.6 and 3.1 installed
October 30, 2009 at 7:31 pm
@Mitch
You’d have to install an old version of Python to get rid of the warning, or rewrite the script using the new module. Regardless, it’s just a warning. It’s not stopping it from working, it doesn’t need fixing. It’s purely informational.
October 31, 2009 at 11:01 am
HELP!!!
I have been reading your blog for 2 days and have tried everything that you have suggested to your readers. I have tried the advise and step by step of other posts. I have never tried something like this and am pretty sure that somewhere along the way. I have done something wrong or am leaving out a step. Please help. I have 249 ebooks from Mobi and would love to be able to convert them to a different format.
Thanks
October 31, 2009 at 11:42 am
Ah…Revx thanks..unfortunately am still doing something wrong. Sorry to bother you,but I suspect this is something simple in the syntax as it keeps telling me the creditcard or name is wrong.
This seemed to me as it was in the script
ereader2html.py book.pdb \”Joe Blow\” 12341234″
Also tried the usage syntax that appears on the screen at times and not in the script of
ereader2html.py book.pdb “Joe Blow” 12345678
None of those seem to work. On one occasion I got it to actually create a directory (Joe) but then it told me the name and credit card number were wrong. Re-downloaded file to make sure of that, same result.
I’m sure this is simple, unfortunately something is not clicking. Sorry for the time waste.
October 31, 2009 at 11:54 am
@Cyndi
Try joining Mobileread and asking nicely there. I’m sure someone will send you a private message and help out.
Don’t forget to mention what operating system you’re using (Windows, Mac or Linux)
October 31, 2009 at 1:19 pm
Thanks for the hints on eReader2html and footnotes. I would be happy to take a look at this one and try to add support.
I have added:
self.num_fnote_pages = struct.unpack(‘>H’, r[46:26+2])[0]
self.first_fnote_page = struct.unpack(‘>H’, r[44:24+2])[0]
and modified getText to simply dump any footnotes out after the main text
if self.num_fnote_pages > 0:
for i in xrange(self.num_fnote_pages):
r += zlib.decompress(des.decrypt(self.section_reader(self.first_fnote_page + i)))
and in a vain attempt to get internal anchors and links for footnotes done the following:
def ilinkPrinter(link):
return ‘‘ % link
and added to html tags this entry
‘Fn’ : (ilinkPrinter, ‘‘),
but I really can’t test and fix anything without a DRM encoded eReader file to see more.
I would be happy to buy a cheap or free one from someplace if anyone knows of one that definitely has footnotes.
October 31, 2009 at 1:22 pm
Hi,
In case anyone wants to try my changes above, please correct the following:
self.num_fnote_pages = struct.unpack(‘>H’, r[46:46+2])[0]
self.first_fnote_page = struct.unpack(‘>H’, r[44:44+2])[0]
and of course you have to add the proper tabs as well
October 31, 2009 at 4:56 pm
Okay, I have ereader2html spitting out the full text of the footnotes now.
The problem is the 1st record of the footnotes which is 63 bytes in length (in my particular case) is not a multiple of 8. That means that the decrypt routine won’t touch it.
So I tried just zlib decompressing it but I get a zlib header error.
I have dumped the first record but it seems to be gibberish or encoded somehow. Given what Mobi Wiki says, it is supposed to be a null separated list of footnote names but I can’t see it.
Any hints on what needs to be done to the first record of the footnotes would be greatly appreciated.
November 2, 2009 at 2:38 am
Paul,
I appreciate your response to my posting on 10/31. I took your advice and paid a visit to Mobileread. I did a search for past and present conversations pertaining to the topic of DRM removal and all results point back to mobidedrm, python, and that I should do a Google search of these topics. Which lead me back to here.
I have once again read through each posting on this site looking for assistance from the responses given to others that have had similar problems like my own. I have tried the solutions that have been given and unfortunately I can not get this to work for me. If you have the time I would greatly appreciate any advice or assistance you would be willing to give.
I am trying to run mobidedrm06 with windows (Vista) operating system.
Thank you,
Cyndi (cysomers68@att.net)
November 2, 2009 at 4:47 am
Hi Dark Friend,
Thanks again for your hint about handling footnotes. I am now able to get all of the footnotes in readable fashion. The problem is the first page of the footnote information is 63 bytes long (in my test case) and actually is not encrypted. By watching the code run in gdb, I was able to see that the page 1 footnote data is exclusive ored with a sequence of 32 entries from a table so make this useful. I simply can not see where this table of values is coming from. pdbshred hints at a fourth key based on the first 8 bytes of the sha digest of the “title”. I am not sure what he/she means by title.
Any hints or ideas of where the table to clarify the footnote page 1 info would be appreciated.
All of this just to get my copy of Freakonomics to work with my new Sony reader. I guess I could just read it on my computer or my ipod touch but I really prefer the Sony reader so …
Thanks for any hints.
November 4, 2009 at 1:37 am
Kevin — http://pastebin.de/1196
November 4, 2009 at 1:42 am
Kevin – http://pastebin.de/1196
November 4, 2009 at 2:56 am
Hi 123 and Kevin Hendricks Friend,
Thanks that is a huge help!
I figured out that they are walking an XOR table modulo some set length starting at an initial offset into the table (which depends on the relative page number within each section). It took me forever to find that XOR table actually sitting in data of record 1 **before** it is unencrypted and unshuffled. Once I had that I could dump the first footnote ids record and make it understandable, as well as all of the links records (allowing me to determine where they start and the count, and ditto for the chapter records, expanded text sizes record, metainfo records (all for type 272 only).
If you want any of that let me know.
Now I can actually read the footnotes in Freakonomics after converting it to ePUB and reading it on my brand new Sony ereader 600B!!
Take care and thanks for bringing back “fair use” and “common sense” to the messed up ebook industry. There are many people (myself included) that will *never* pirate anything and just want to read the damn things on their platform of choice and be able to archive it for the future when many of these devices are long gone.
November 4, 2009 at 4:48 am
See
http://pastebin.de/1199
November 5, 2009 at 2:40 am
Kevin
Can you provide your final working version of the script? Is there any difference in yours when compared to either of 1166 or 1199 mentionedearlier
November 5, 2009 at 3:38 pm
I looked at both.
1166 is the script to start with since it has many improvement over version 0.03b of ereader2html.py including footnote support (which unfortunately does not handle footnote record 0 properly)
1199 was based off the older 0.03 version but adds features that properly handle footnote record 0 to get a list of the actual footnote ids and using that same approach, is able to properly output chapter offset records, link records, and etc.
Both of these are for type 272.
So people need to start with the generally more advanced 1166 and join with it the new pieces from 1199 (XOR table, properly decoded footnote ids, etc, etc). They could also easily mimic the use the footnote changes to properly handle sidebar changes as well. Note: sidebars may map to the same first record as footnotes but they have a count of 0 if they are not being used and visa versa).
I hope some kind soul will paste a properly merged version to pastebin soon.
November 5, 2009 at 3:51 pm
Whoops!
Everywhere it says 1166 it is supposed to say 1196
November 5, 2009 at 9:55 pm
I’ve combined 0.04 and 0.05 to make a new 0.06.
I’ve made it put the images in a sub-folder, so that you can just drop the PML file onto DropBook and make a new eReader file.
I’ve improved the HTML output a bit, although really you get better results making an unencrypted eReader file using DropBook and converting with Calibre.
I’ve tested on the few encrypted eReader books I have, and it seems to be working well. I haven’t been able to test any of the fancier features.
November 5, 2009 at 9:56 pm
And I’d better mention where to find 0.06!
http://pastebin.com/f76582f9b
regards,
Alf.
November 5, 2009 at 11:30 pm
Here is a slightly different version with 0.04 and 0.05 merged that handles some of the new stuff a bit better and adds sidebar support (sidebars still need to be tested but it should work – theoretically)
http://pastebin.de/1278
Now we need to merge this with the html and image folder fixes from 0.06!
November 6, 2009 at 12:55 am
Okay, I fixed an indentation issue with the newest version 0.05, removed some of the extra debug info from it, and added in the image folder and html fixes and things from 0.06, so we now have a 0.07 if anyone wants it, that should have everything to date in it.
http://pastebin.de/1280
November 6, 2009 at 4:51 am
Does anyone know of python or perl (or even java) code that is smart enough to split html files intelligently into many smaller html files (and by intelligently, I mean not splitting in the middle of a tag, properly updating any internal links to point to the achor that may now resides in another html file, etc.
I really don’t like the code generated by calibre going from html to ePub. It seems to miss all toc entries, and many internal links are not being updated, and there are so many styles added the resulting xhtml is almost unreadable which makes hand editing hard.
So I thought I would look to see if there is something better and thought people here might know of something.
Thanks,
Kevin
November 6, 2009 at 8:12 am
For hand-editing (X)HTML to make ePubs, take a look at Sigil.
http://code.google.com/p/sigil/
The GUI for HTML markup is still in the early stages, but you can easily edit the (X)HTML code directly, and it does a very good job of generating correct ePubs.
November 6, 2009 at 3:42 pm
Thanks for that sigil url. I just tried it and it took about an 45 minutes to manually set chapter breaks and table of contents, and meta information and then it generated a very nice epub file.
It seems ereader2html does not automatically add around chapter titles so Sigil will not recognize any TOC entries. Nor does Sigil parse the meta tags that are already in the html input file. So I had to reenter them manually.
Other than that, the epub it created was very very good. Sigil very intelligently handled all internal links (even those for footnotes) when it split the files. Manually unzipping the created epub file shows a well structured set of files and directories.
I am very impressed!
Sigil will now replace calibre for my html to epub conversions.
Thank you.
November 6, 2009 at 10:41 pm
Hi RevX
I have downloaded the latest eBookUtility. I have Java and Python installed on my computer. When I double click on eBookUtility it asks for the serial number (of my Kindle, I assume).
I enter the serial number, taken directly from the back of my Kindle, and it says that it “Cannot generate PIN. Invalid Serial Number?”
Any advice? My Kindle is a Kindle 2 International. The Serial number is definitely correct. I even know its PID.
Many thanks. I want to use your application to strip a Mobipocket ebook of its DRM so I can read it on my Kindle 2.
Cheers, Will
November 6, 2009 at 10:51 pm
It seems that eBookUtility doesn’t yet understand International Kindle Serial numbers. The decode just like otherKindle serial numbers, but they start with B0003. (Is that the right number of zeros? anyway – where Kindle and Kindle 2 have a 1 and a 2 the Kindle International has a 3)
November 6, 2009 at 11:05 pm
I see. Thanks Paul. You’re almost correct re the serial number, its B003.
I’ve also tried the command prompt kindlepid.py and kindlefix.py scripts with a PID that I know is correct for my Kindle. However the kindlefix won’t recognise the PID that kindlepid.py generated, that i know is correct. I’ve been confused as to why, but perhaps your response explains it now. I guess living in Australia means I’m not going to be able to strip these mobi files to be read on my Kindle. Disappointing, but thanks for the information.
November 6, 2009 at 11:16 pm
You don’t need eBookUtilty to strip DRM from Mobipocket files. Just an Installation of Python 2.6.4, and the MobiDeDRM 0.06 script. Python you can get from python.org, and the script is probably available from one of the links in previous comments.
November 6, 2009 at 11:21 pm
Thanks Paul, but the MobiDeDRM script doesn’t recognise my PID…even though it definitely is the PID of my Kindle (I’ve double checked it through the MobiPocket Reader 6.0).
Perhaps, as you’ve indicated in your previous post, this is because my Kindle is an international version.
November 6, 2009 at 11:30 pm
MobiDeDRM does work with International Kindle PIDs.
Most likely, you just need to surround the PID by single quotes or double quotes on the command line (I forget which ones work on Windows).
November 6, 2009 at 11:34 pm
It does huh? Well I wonder what is going on. I have tried the single quotes and double quotes trick before on the command line. But alas, no joy, it still says the PID is not correct. Thanks for your advice, i’m resigned to the fact that MobiDeDRM just isn’t going to work for me.
November 6, 2009 at 11:43 pm
Feel free to email me off-blog if you’d like to. I hate to see someone stuck with DRMed ebooks. See my website.
November 7, 2009 at 2:59 am
Thanks so much Paul. Just sent you an email.
November 7, 2009 at 5:12 am
This is a response to Will’s message earlier.
If I understand things correctly, you have purchased a DRM protected ebook in mobi AKA prc or mobipocket format. If this is the case then the ebook was encrypted with a PID that was previously registered with the ebook vendor. mobiDeDRM should work with the PID registered with that vendor. The PID associated with your Kindle is irrelevant and in fact, mobiDeDRM will report back to you that your Kindle PID entered into the script in incorrect.
I think you mentioned in one of your earlier posts that you have the MobiPocker reader installed. I’ll bet the PID associated with your instance of that program is the one that should be entered into mobiDeDRM.
Again, referencing an earlier post, the double quote markes surrounding the PID won’t hurt but should not be necessary unless you have some special characters that are meaninful to DOS
November 7, 2009 at 7:26 am
Yes, Will, the Kindle PID will be useful for stripping the DRM from Mobipocket files purchased from Amazon for reading on the Kindle. It won’t be useful for reading books from other sources on the Kindle, unless that’s the PID you registered with the vendor when you purchased the file (vendors who sell DRMed Mobipocket ebooks, and libraries who check out Mobipocket ebooks, make you enter the PID of your reader or device before you can download the book; that PID is used to infest the file with DRM the file and the DRMed file is what you can download.)
November 9, 2009 at 5:15 pm
You’re my hero! Now I can read those books that I bought for my Palm on my much nicer Cybook! Thanks.
November 10, 2009 at 9:12 pm
Please see
http://pastebin.de/1362
This is a trial version 0.09
This one tries to improve support for pml to html conversion by borrowing from calibre’s pml to html conversion approach as a starting point
- uses appropriate xhtml tag pairs everyplace
- replaces deprecated html with css pieces
- adds an internal only pseudo tag \h to handle the case where the \t tag is used to generate hanging indents
- properly handles the remaining \t tags which do block indents
- uses html comments to mark ChapterTitle Starts which makes it easy to generate Tables of Contents using Sigil especially when the toc entries are not found by Sigil automatically
Pretty much everything is reasonably handled except for:
- the \T=”40%” tags which are really tab stops that can not be easily incorporated into html without setting up tables of some sort (which are not supported by most ebook readers)
- the normal and standard fonts tags
November 10, 2009 at 9:40 pm
These sound like really useful improvements for those just interested in the HTML output.
However, I’m not sure that it’s worth spending much more time on the html conversion part of the script. It’s simple (from 0.06 on) to create an unencrypted eReader file from the output using DropBook, and then do any necessary conversions using that file through Calibre.
I don’t mean to be negative about improvements to the script, but might it be more use to more people to spend the time on improving the Calibre eReader import routines?
November 10, 2009 at 10:28 pm
Hi Paul,
The Calibre ebook-convert routines simply converts pml to html (Calibre’s intermediate form) and then from html to epub (or whatever version you want). Unfortunately, it makes html code that is very very hard to read at times and does not gracefully handle table of contents and the like, so you end up with epub files that are sub-optimal.
So even though you use something like Dropbook and then Calibre, all you are really doing is converting the exact same pml to html inside a python script inside of Calibre and then html to whatever other format you want.
I would much rather skip Calibre completely (which is way overkill for doing html to epub conversions), and use Sigil to convert html to epub.
It simply does a better job os slicing the html into correct portions, creating a better table of contents, etc, without all of the overhead of
Calibre.
As for long-term archiving, I would much rather have an html version than a pml version. They way you use it, ereader2html is really DeDRMereader but for me archiving pml is not a long term option.
Therefore, I would like a much better pml to html converter and that is what I tried to do above.
My 2 cents …
November 10, 2009 at 10:58 pm
Fair enough. You’re putting the work in to make the tool you want, and being kind enough to share it. Thanks.
November 11, 2009 at 10:22 am
thanks, guys, you are all really great and this “Reply section” must be the most useful one in any blog I know and care
I’m even starting to learn Python now, after all those technical discussions you had here leaving me wanting to understand more about all this new programming languages (I was once a profissional programmer, but with some old languages like Cobol, Pascal and Fortran
– I know some VB and C++, but I want to “modernize” myself 
)
so, thanks again for all the work on this tool!