Other Weblog Headlines

Using RDF Summary, a DTML/COREblog module, a python script and some duct tape I made another info box. This one has the newest 3 headlines of some weblogs I find intersting. Updates should happen once a day, as these are not very active blogs.

Problems were mainly to get the formats to work, since RDF Summary has to be told in the source code about tags it does not know yet. Also scuds blog does not emit "date" information for blog entries, which produced an error (that is now caught).

Two COREblog patches coming soon

Today I fixed up two little patches for COREblog (the software behind
the blog you are currently reading). One is an improvement of the
"moblog" (mobile blog) feature, I've added Subtitle support. Couldn't
help it, I'm a Subtitle freak. The other one is a fix/hack for a bug
with Trackbacks. Trackbacks appeared when you clicked on them, but the
Trackback-Count was still listed as zero. Detailed information, patches
and instructions follow after some more testing.

Patching Moblog to Support Subtitles

This procedure will allow you to add entries to your blog with COREblog's "moblog" feature and get a "subtitle" line too. Out of the box "moblogged" entries can just have a title (which is transmitted in the mail's "Subject" line) and picture (attached image, automatically added to the start of the entry). With the patch, you will have Subtitles for your moblog entries too. And yes, WARNING: This works for me, but who knows if it messes up everything for you. Test thoroughly if it works for you and if you don't know the stuff about getting it to work, better ask someone to help you with this.

After the change, the mails you send to to moblog should look like that:

Subject: Title for the Entry Goes Here

password123              <- first line of message, password
digital                  <- 2nd line of messsage, category
Subtitle will be here!   <- 3rd line of the message
And starting right after that is the text of the entry...

There is no switch for the change. If after applying the patch you don't want to have a subtitle for a post, you will have to add an empty line in the proper place. Otherwise the first line of your post will end up as the subtitle.

The patch

Save this to a file "moblog_subtitle.patch" in the Products/COREBlog directory of your Zope instance:

- - - - - - - - - Cut here - - - - - - - - - - -
*** COREBlog_old.py     Wed Jan 26 15:58:24 2005
--- COREBlog.py Wed Jan 26 16:18:10 2005
***************
*** 1543,1552 ****
                  pict_attached = 1
  
              #add entry body
!             if len(body_l) > 1:
                  blog_charcode = self.getProperty("management_page_charset")
                  posting_title = mh.getSubject()
!                 posting_body = join(body_l[1:],"\n")
                  if pict_attached:
                      posting_body = """<img src="./images/%s" border="0">\n""" % (pictid) + posting_body
                  main_category = self.getProperty("moblog_default_category")
--- 1543,1553 ----
                  pict_attached = 1
  
              #add entry body
!             if len(body_l) > 2:
                  blog_charcode = self.getProperty("management_page_charset")
                  posting_title = mh.getSubject()
!                 posting_subtitle = body_l[1]
!                 posting_body = join(body_l[2:],"\n")
                  if pict_attached:
                      posting_body = """<img src="./images/%s" border="0">\n""" % (pictid) + posting_body
                  main_category = self.getProperty("moblog_default_category")
***************
*** 1564,1570 ****
                  new_id = self.manage_addEntry(self.getProperty("author_for_moblog"),\
                                       posting_body,"","",\
                                       main_category,1,sub_category,\
!                                      posting_title,"","",0,comment_status,trackback_status)
                  if pict_attached:
                      try:
                          #set metadata(title,permalink) to added image
--- 1565,1573 ----
                  new_id = self.manage_addEntry(self.getProperty("author_for_moblog"),\
                                       posting_body,"","",\
                                       main_category,1,sub_category,\
!                                      posting_title,\
!                                      posting_subtitle,\
!                                      "",0,comment_status,trackback_status)
                  if pict_attached:
                      try:
                          #set metadata(title,permalink) to added image
- - - - - - - - - Cut here - - - - - - - - - - -

Watch out for any stray line breaks when copy/pasting!

Applying the patch:

This is a multistep process, involving changing the COREblog source code in your Zope's "Product" directory, let's go step by step:

Backup the original file: Move to the "COREblog" directory and there do:

$ cp COREBlog.py COREBlog.py_backup

Apply the patch: With the patch file in the same COREBlog directory, do:

$ patch -p0 < moblog_subtitle.patch

Output should look like that:

Hmm...  Looks like a new-style context diff to me...
The text leading up to this was:
--------------------------
|*** COREBlog_old.py    Wed Jan 26 15:58:24 2005
|--- COREBlog.py        Wed Jan 26 16:18:10 2005
--------------------------
Patching file COREBlog.py using Plan A...
Hunk #1 succeeded at 1543.
Hunk #2 succeeded at 1565.
done

Refresh the COREBlog Product: Navigate in the Zope Web Interface to the ControlPanel/Products/COREBlog screen, click on the "Refresh"-Tab and there on the "Refresh this Product" button. The changes you made will now be available. How refreshing!

Use this at your own risk and as much as you like, feel free to give my patch away and incorporate it into the COREBlog source if you are the maker :-)

Zope, Formulator, Localization, UTF-8, and... searching for a Solution

Sometimes searching for a solution to a technical problem on the web is an endless stretch of pain. I was working on a small Zope project, when I ran into a problem with... (who would have guessed it) UTF-8 encoding. I had that project set up just like it should be done (Zope Page Templates (ZPT), Python, Formulator, Localizer, TranslationService), everything was working nice and dandy. And then along comes a small problem that turns out to be a real showstopper. Spent hours and hours searching. In those situations I just can't give up. When I finally found the solution, long after I had gone home, I promised myself to write a solution report on my page, so others will find it. Stay tuned.

Zope, Formulator, Localization, and the Solution to Translating Option Values

So you set up a nice little form with Formulator on Zope? And you went ahead and provided multilingual content with Zope Page Templates (ZPT) and TranslationService, translating all the text snippets on the page and even the form element titles and texts with the i18n:translate syntax. Well done. At least until you get stuck on the "option" values in "select" fields that you have done in Formulator. How do you translate those? There is no place to put the i18n-attributes in your ZPT and you don't want to make the form manual again after setting it up with Formulator. Read on for the solution:

Older versions of Formulator seem to have had some hook for i18n-Translationservice, but according to the HISTORY.txt-file that's gone (with only an obscure remark that it can be better replaced with i18n:translate - which you are already using, except for the option values). It's really OK for this to be gone, but some more info for the solution is necessary.

What you need is the "TALES" tab in the Formulator management screens. For the field in question, you have to access the Localizer-translation with the TAL "python:" syntax. Provided you have added a MessageCatalog called "MessageCatalog" (just like you have to set it up for the TranslationService), you would use:

python:here.MessageCatalog.gettext('Name of text in MessageCatalog')

A short explanation might be needed if you don't have much experience with MessageCatalogs and translation through Localizer: What happens is not really that the text gets "translated". The idea is that the MessageCatalog holds text snippets in multiple languages for a project. The "names" of the text snippets might be the original text in one language ("Thank you for providing us with this information") or they might be an abbreviated identifier for the text ("thank you note bottom"). The "gettext" method and the ZPT/TranslationService "i18n:translate" syntax do a lookup in the MessageCatalog and return a suitable string for the "current" language. A string which you or someone else has entered into the MessageCatalog.

This TAL/python example works well for Text fields, but fails for any kind of list field. The reason is that Formulator expects a python list as a result, and what it gets is a string. So it turns each character into a list item and in your popup menu you can choose such informative items as "b" instead of "blue".

My solution was to make a python script (called "translateme"), which contains:

Parameters: snippet='default option list'

import string
return string.split(context.MessageCatalog.gettext(snippet),'||')

The script is called from the "TALES" tab of Formulator with:

python:here.translateme('optionlist')

As you noticed I've set up one parameter for the script. I'm using a default value, because most option lists are the same for this project. But you don't have to use that obviously. Then I get the translated text snippet out of my message catalog. In the MessageCatalog, I use "||" to separate the items of the option list. The "string.split" part in the script is responsible for making a list out of that, just like Formulator expects.

So, now I hope this is helpfull for someone else running into this or a similar problem. At least it will serve as a reminder for myself :-)

Experiments with RAM cache

I've set up a Zope RAMCacheManager for caching some of COREblogs elements in RAM. Will see how well it works with updates and how much performance will improve.

RAMcache for COREBlog update

More hints for setting up Zope's RAMCacheManager with COREBlog and especially the entry_body dtml-method. So far my wisdom is that for "Names from the DTML namespace to use as cache keys" in the "cache" tab I need to enter "id" and "noextendlink". Without "id" all entries will get rendered as the same story, without "noextendlink" the extended, full view of the story is the same as the "lead" on the weblog page. I don't know yet how caching will behave with comments (because there aren't any so far...), but chances are that trackback and comment counts should be added to that list too.

Problems with Comments Solved?

Just fixed an issue with permissions that prevented anyone not logged into Zope from adding (or even previewing) a comment. Hopefully this solves the problem. Comments are still moderated though. Maybe this was why there were no comments so far. Thanks w0lfshade!

ZopeTree Using SESSION patch

Dug around with ZopeTree (also ZopeTree on Sourceforge). Managed to get something working (despite almost total lack of documentation), except I had a problem with cookie paths in different browsers messing up the Tree state. Decided to use Zope's session engine. Here is a patch for ZopeTree that seems to make it work: zopetree_session.txt

Updated: I'm not using ZopeTree any more, instead I use ExpansionTree, see "Trees for Page Templates"

Search in Greek Works Now

Zope splitters are: a.) used to split text into words to make them indexable and b.) seemingly deep magic because documentation is nonexistant (or I couldn't find any). What I ended up with was the sourcecode to CJKSplitter. So I hacked CJKSplitter into doing Greek. It was not especially difficult, mostly a "delete" job (chinese is way more complex). Also I replaced the range of chinese unicode characters with the range for Greek. Might add iso-8859-1 character ranges too, especially for this blog.

Once I clean up the code, have a look at the license and ask on the mailing list if this is really the proper way to go, I will put this online. It might be a good solution for Greek in Zope with ZCTextIndex.

But in short: The result so far is that searching in Greek should now work in the search form on this weblog.

Some Links for Searching and Greek

Although this is not a "link blog", here are some references I found while fooling around with splitters for ZCTextIndex.

This is somehow the same problem I'm trying to solve, someone who wants to search using ZCTextIndex and Unicode characters:

http://mail.zope.org/pipermail/zope/2003-June/137115.html

Here are some greeks who want to search in iso-8859-7, an old entry into the Zope Collector:

http://www.zope.org/Collectors/Zope/842

(I'd suggest switching to UTF-8 and using something like the splitter I made with ZCTextIndex.)

The chapter about searching and indexes in Zope:

http://www.plope.com/Books/2_7Edition/SearchingZCatalog.stx

Unicode character table:

http://jrgraphix.net/research/unicode_blocks.php?block=7

Had to do this on my source file to stop a warning (because python wants to know about unicode characters in source files):

http://www.python.org/peps/pep-0263.html

This should really resolve all these issues and provide for better search functionality:

http://zopyx.com/OpenSource/TextIndexNG/

(I could not find any documentation about e.g. the supported languages on the TextIndexNG site, and I would have to change COREBlog, so I gave up on this for the moment.)

Erfahrungen

Als ich Ende letzten Jahres wieder einen Job suchte, erwartete ich nicht, dass ich wieder etwas mit Zope machen könnte. Zope-Jobs sind nicht wirklich weit verbreitet. Aber mein Glück, das Schicksal oder Wasimmer wollte es wohl so, seit einiger Zeit bin ich wieder mit Zope am werkeln und sammle weitere Erfahrungen.

Bei meinem letzten Zope-Job (network ag) konnte ich viele schwierige Passagen an meinen Kollegen und versierten Zope-Meister Gidon Friedman weitergeben. Das ist hier nicht mehr so, da ich der einzige Programmierer im Haus bin. Also habe ich inzwischen gelernt von Null auf meine eigenen Zope-Produkte zu schreiben. Schwierigere Themen spare ich mir manchmal etwas auf, bis ich noch ein Stück mehr dazugelernt habe.

Momentaner Knackpunkt ist ein Zope User Folder, mit dem ich für die einzelnen Benutzer auch zusätzliche Properties verwalten kann. Da gibt es die Möglichkeit das mit exUserFolder zu machen, bei dem kommt schon so einiges fixfertig mit. Die zweite Möglichkeit ist SimpleUserFolder zu verwenden und darauf aufbauend einen eigenen UserFolder abzuleiten. Diese Variante liegt mir mehr, da ich 99% der Komplexität von exUserFolder nicht brauche.

Nur Schreiben muss ich das Teil noch. Bei meinen ersten Versuchen bin ich recht schnell hängen geblieben: Dokumentation und Handbücher sind bei diesen Tools nicht gerade füllig vorhanden. Und bei vielen Howtos geht es genau um die Dinge, die mich nicht interessieren, wie Anbindung an SQL-Datenbanken. Beste Resourcen, die ich bis jetzt gefunden habe:

• Beispiele für einfache Methoden zu SimpleUserFolder (in DTML, aber da kann man ableiten)
• SimpleUserFolder with Zeus and PostgreSQL bei ZopeLabs
• MARC mailing list archive um auf der Zope-List zu suchen, was besser funktioniert als mit einer WWW-Suchmaschine.

Zope: Using UTF-8 in the Management Interface (ZMI)

To really use utf-8 unicode in your zope management interface (as of Version 2.7.X), go to the "Root Folder", click on "Properties" and add a property with the following settings:
Name: management_page_charset
Type: String
Value: utf-8

Why is that necessary? Why does it seem to work without it too? It somehow does, but when entering unicode into text fields, those characters will get replaced with html entities like {. Confuses ZCatalog searches and is impossible to edit.

Update 9.5.2005: One pushback with this solution is the handling of properties in the ZMI: On the properties tab all the choices for Unicode types ("ustring", "ulist", etc.) are gone. Even when I add properties with those types through a script, the display and editing of such properties is broken. I think some of the trouble is documented in this Zope Collector entry called " Zope unicode patch". But I believe there might be other problems too.

Trees for Page Templates

After my last note on using ZopeTree (with patch), I had contacted the author and heard that ZopeTree will be all new and nice in Zope 3. But for the moment the lack of documentation made me look elsewhere. What I'm using now is ExpansionTree.

Expansion Tree has just a little bit more documentation. But it works nice for me, especially for storing state information in the SESSION. The tree generation logic is not in my Page Template, but in a method of my Python Product. Getting the state is done with a simple:

ctree = request.SESSION.get("ctree", '')

and after processing we can store the state again, approximately like this:

[...some other code...]
tree, rows, state = t.grow(root, ctree,
        request.form.get(todo_var, ''), default_state={})
request.SESSION["ctree"] = state
return {"tree":tree, "rows":rows}

Where the "return" will give back the values needed to display the tree in the ZPT (Zope Page Template).

Caveats: When the SESSION expires, the tree state is gone and the tree will be shown collapsed again. In this application I have set the SESSION to a long time, plus I have the tree in a default partially expanded state. But you may want to look into this.

zope haikus

23.6.2005 on #zope:
< __mac__> woodt: you are on a cut/paste action when a manage_afterAdd is
                 called after a manage_beforeDelete
< __mac__> so you have to remeber in manage_beforeDelete that there might
                 come a manage_afterAdd after that
< betabug> that just sounds so philosophical
<@TheJester> Very Zen.
< betabug> so beforeDelete
< betabug> winter may go spring again
< betabug> manage_afterAdd
< betabug> 5-7-5, season reference -> complete haiku
<@TheJester> depends if you pronounce the underscore or not d8)
< betabug> definitely not, I never do (honest, not just in zope haikus)
< betabug> but changing the first line to "think beforeDelete" would be
          an improvement, though violatiing the "spirit of the moment"
...
<@TheJester> call beforeDelete / remembering spring may call /
                   manage_afterAdd

So, this sounds really nice to me:

So beforeDelete
winter may go spring again
manage_afterAdd
-- betabug 23.6.2005

Or, if you like TheJester's version better:

call beforeDelete
remembering spring may call
manage_afterAdd
-- TheJester 23.6.2005

Which in my oppinion floats nicer from line to line. Also makes more sense in a zopeish way.

Comments? Additions? Flames?

Plone: Getting NavTreePortlet to Display Always 2 Sublevels

Yes, another of those boring technical posts. Trying in Plone to display a NavTreePortlet with always the first and second level of the navigation tree shown. Plus the current page highlighted. Found a hackish solution, but it may not be 100% what other people need. Basically we are switching to "sitemap" mode, set the sitemap depth to 2, and patch NavTreePortlet to highlight the current page in sitemap mode too. Here it comes...

First step is to switch to "sitemap" mode. So customize "portlet_navtree_template" and change the first div with the tal:define of "data" as such:

<div class="portletContent odd" 
tal:define="data here/getSitemapData">

Now go to settings of the NavTreePortlet and change "Depth of sitemap" to "2" (or whatever you like). I believe this will not show sublevels below that, but I may be wrong, haven't actually tested.

This will change the display of the navigation tree a lot, but the downside is that the current item is not highlighted any more. (The current item gets another css class assigned, so by changing that in ploneCustom.css we can alter the appearance of the current item.) We need a fix in the actual code of the NavTreePortlet:

*** NavTreePortlet.py_orig      2005-06-26 08:52:49.000000000 +0200
--- NavTreePortlet.py   2005-06-26 08:48:23.000000000 +0200
***************
*** 54,59 ****
--- 54,61 ----
          if context == self or sitemap:
              currentPath = getToolByName(self, 'portal_url').getPortalPath()
              query['path'] = {'query':currentPath, 'depth':self.getSitemapDepth()}
+             # trying to fix detection of current item in sitemap mode
+             currentPath = '/'.join(context.getPhysicalPath())
          else:
              currentPath = '/'.join(context.getPhysicalPath())
              query['path'] = {'query':currentPath, 'navtree':1}

Apply this patch to NavTreePortlet.py in the INSTANCE_HOME/Products/NavTreePortlet directory with something like

patch -p0 < navtree_sitemap_highlight.patch

As usual this change is at your own risk and you better know what you are doing :-)