We are currently planning to migrate to a different platform for the forum. We will announce this fact both here and on Twitter, once we have settled on a solution.

Zotero as Zettelkasten?

Hi, I am a Zotero user of many years and once I came across Zettlr I realize that I have been using Zotero as a Zettelkasten of sorts without being familiar with that term or concept. I like Zettlr and would like to switch to keeping my notes in it, primarily to have a future-proof notes storage system, and to keep my Zotero database more manageable by removing the many notes there. But my current system seems to have a lot of advantages over Zettlr's current functionality. Can the developer tell me whether they considered doing the Zettelkasten in Zotero and if they did, why they rejected the idea?

I am also hoping people can comment on my existing workflow and some of the advantages I currently see for Zotero so I can see if these are not really advantages, if Zettlr has advantages I don't see, or if there are other considerations. This isn't an attack on Zettlr, of course - I'd like to move to it if possible.

Zotero is able to do many things required by a Zettelkasten:

  • store notes either as Zotero notes (integrated into the main database and editable using a SimpleHTML interface window) OR as plain-text files as attachments to Zotero items (which is what I do, and I edit them in markdown in Notepad++)
  • each attached notes file is given a unique identifier, is full-text indexed for search, can be "related" (strong-linked) to any other items, can be tagged with many tags (weak links), and can have additional notes added to them as metadata (easily visible in the Zotero window)

Zotero's apparent advantages:

  • most importantly for me, customizable full keyboard navigation. The design of Zettlr strongly suggests to me that the developer is not a heavy keyboard shortcut user, which is a shame because I cannot tell you how much of a timesaver it is, especially when doing tons of repetitive actions as you do in research. With keyboard shortcuts I can instantly highlight the library, tags, or item windows without having to tab through a lot of selections or use the mouse.
  • notes are attached to their bibliographic items, so if I'm looking for a note or idea, I can use methods other than searching for a known tag, such as sorting by date added, author, item type (book/article), etc.
  • notes can be grouped not just by tags, but also folders (categories), sub-folders, and saved searches (so I can have a folder showing all notes tagged "emotion" based on articles (not books) created in the last 30 days)
  • the tag system is much more robust
    • can assign 9 tags unique colors and hotkeys for quick use
    • can see all tags or just tags used by the items in a category/folder
    • can visually scan all tags in the resizeable tag window faster than you can scroll through a list

One other advantage of full keyboard navigation is that it allows me to write AutoHotKey scripts to perform more complex repetitive tasks, such as creating a notes file for a source, with one keyboard shortcut that does this:
1. Attach a plain-text file that is a notes template including lines for tags, links, etc.
2. Copy/paste the full citation of the item AND a direct link to that item in Zotero (using the zotero// URL scheme) at the top of the note
3. Close that notes file
4. Rename the attachment in Zotero with the author's name and date

There are other advantages, but I'll stop there for the time being.

Not having used Zettlr much, the advantages seem to be in these categories:

  • less resource-intensive, more lightweight, with faster (but less robust) search
  • advantages in editing notes: Markdown style rendered on-the-fly, foldable headings, Table of contents selector

Thanks for any response.

Comments

  • Thanks for the long comment of thoughts concerning Zettlr and Zotero.

    While some of this information certainly holds true, in the past few weeks, a lot of changes have made their way into Zettlr that directly address some of these issues.

    First to Zotero as a Zettelkasten system: Zotero has been built as a reference manager, that means, while being able to curate a Zettelkasten, it was not the main intent, and certainly requires a certain amount of hacks to make it efficiently. Zettlr, on the other hand, has been built by me to precisely complement Zotero. Maybe it helps to elaborate on my workflow:

    1. Find an interesting paper
    2. Download it and add it to Zotero
    3. Read it using my PDF viewer and annotate interesting things
    4. Extract the annotations into a linked note
    5. Copy and Paste these into a new "Excerpt" in my Zettlr folder
    6. Split them up using a script into individual Zettels that I then edit

    What has been added to complement this kind of workflow (and, hopefully, others) is a stronger cross-integration with Zotero, meaning: You can now cite items from your database and Zettlr gives you direct access to the correct item in Zotero using a right-click on the citation). Therefore you have the cross-link.

    Concerning search: Well, it's work in progress, because currently the underlying algorithm is inefficient (about 10,000 files take almost 5 seconds on my state of the art computer, which is at least 4 seconds too long), and then the search itself also needs some retrofitting, certainly. PRs on this are welcome.

    And concerning the keyboard-stuff: Yes, I'm a macOS user and the mouse support on macOS is thousands of times better than on any other computer, because you can actually do a lot of time-saving stuff using the mouse on macOS. But it's not quite right that I'm not a heavy-keyboard user. In fact, I also don't like to use the mouse for a lot of stuff, so there is quite a decent amount of functionality already accessible via shortcuts. See here: https://www.zettlr.com/shortcuts

    The note-grouping is actually quite well in Zettlr, with not only using tags (Zettlr detects that it should search for tags, you just have to prepend a #) but you can also create outlines (the counterpart to folders). And, it used to be possible to use virtual directories, but due to a hefty load of new features, the virtual directories functionality has become unstable and I need to tackle this next.

  • Thanks for the response. Hope my comment wasn't too long! I'll split my comments into sections.

    Zotero

    Can you explain how/why Zotero requires hacks to act as a Zettelkaster? It may help if you tell me how your workflow proceeds after step six when you are actually combining the notes to write a paper.

    Re: annotations, are you using Zotfile to extract annotations in step 3? My workflow is like yours for 1-2, and then:
    3. Read the pdf, highlight key quotes, and add comments in the pdf
    4. While reading, take notes on the source in a plaintext file that I later attach to the item in Zotero
    5. Use Zotfile to extract pdf annotations into a Zotero note, which creates a note that includes:

    • any highlighted text in the pdf reproduced in italics
    • a clickable link between this text and that location in the original .pdf
    • my comments in plain text
    1. Attach my plaintext notes to the Zotero item

    I wonder if you could incorporate Zotfile into Zettlr because it seems to automate some repetitive actions.

    Re: integration with Zotero, I'm eager to try it but (as I commented in another thread) I can't get it to work. I wonder if you are testing with large enough libraries. You are a young guy and it may seem that 700 references is a big set. I'm a much older guy and my Zotero library has 14,000 references. If you hang out on the Zotero forums you will see that it is common for people to have libraries with over 10,000 references.

    Keyboard shortcuts

    Alas, I am a Windows user. I really hope you are open to adding more keyboard functionality to Zettlr. I understand that it seems to you like you are providing a large amount of functionality via shortcuts, but to me (who relies extensively on shortcuts) the list is quite small. Zotero includes 8 functions easily accessible by shortcuts (Edit|Preferences|Advanced|Shortcuts) but has APIs allowing many more. If you do not use the Zotero add-on Zutilo, check it out to see 20+ functions allowed by shortcuts. And if you don't use AutoHotKey, I can't tell you how effectively it can automate repetitive functions. I'm sure if I switch to Zettlr and it has more keyboard accessibility I will be able to come up with AutoHotKey scripts to perform super-fast complex searches, tagging, etc. (If I were a real developer I will be happy to contribute code to Zettlr, but unfortunately, auto hotkeys scripting is at the limit of my abilities.)

    Other

    When you are talking about outlines, are you talking about structured notes that have as content a list of organized links to other notes, and thus act as an outline? Or are you talking about the left "Files" pane? Can you explain how you see Zettlr operating as a method for outlining writing that allows easy reorganizing, etc.?

    Thanks!

  • Can you explain how/why Zotero requires hacks to act as a Zettelkaster?

    Well, it's not really a "hack". But two points I'd take against using Zotero as a Zettelkasten. First, it's meant as a reference manager, but second, and more importantly, the notes are baked into the database of Zotero, as opposed to a plain Markdown approach, which also enables you to switch software depending on what your needs are. It's simply much more "open" than Zotero in that regard.

    It may help if you tell me how your workflow proceeds after step six when you are actually combining the notes to write a paper.

    Well, I usually have my paper open, and then search through all notes in my Zettelkasten, follow links etc. pp. This is much quicker than doing it in Zotero. But, more importantly, this way I can mentally divide reference management and writing to a better degree, but this might only be my personal preference.

    Use Zotfile to extract pdf annotations into a Zotero note

    I do the same, but I have modified the output by Zotfile in the deep settings (e.g. the fundamentals in Zoteros core), so that I can simply copy and paste them and have Zettlr automatically convert it to Markdown. With the corresponding links, I can then quickly switch back to the PDF, if I need more context information.

    I wonder if you could incorporate Zotfile into Zettlr

    Certainly not directly, as Zotfile has been written against Zotero's API and not Zettlr's. Nevertheless, what methodology are you talking about that Zotfile has that does not pertain reference management?

    I wonder if you are testing with large enough libraries.

    I have once had a 10k element library, and it worked out fine. What the most common problem has been in the past were non-allowed characters in the citekeys, because sometimes Zotero produces weird keys with brackets in them (which are, according to the Pandoc documentation, not allowed in citekeys, and this for very good reasons). However, since 1.5, Zettlr should tell you exactly which key was wrong, if it stopped loading your library. It's most probable something like this. Keep an eye out for the notifications in the top-right corner of the main window ;)

    I understand that it seems to you like you are providing a large amount of functionality via shortcuts, but to me (who relies extensively on shortcuts) the list is quite small.

    Well, the thing is: I'm the only dev, so my time is strictly limited, and currently there are a lot of issues open, and I have to weigh the needs and prioritise. Keyboard shortcuts, so far, have not been the sole focus (albeit I've incorporated a lot that were deemed necessary by users!) so if you see a function that has no shortcut where you'd like a shortcut, I'd strongly recommend you open an issue over at GitHub, because then I have it on my list and can get going! You don't need to be a developer, simply have a look at the shortcuts of Zettlr, find some that are good (and cross-platform!) and propose them!

    When you are talking about outlines, are you talking about structured notes that have as content a list of organized links to other notes, and thus act as an outline? Or are you talking about the left "Files" pane?

    The first one. The file list also does have some organising functionality, but mainly structured outlines, yes!

    Here some screens:

    Index file

    Note structure

  • Thanks for taking my comments in the constructive manner they were intended!

    Well, I usually have my paper open, and then search through all notes in my Zettelkasten, follow links etc. pp. This is much quicker than doing it in Zotero. But, more importantly, this way I can mentally divide reference management and writing to a better degree, but this might only be my personal preference.

    It was an interesting surprise for me that you separate searching/managing ZK notes and writing drafts. You do the writing in a regular word processor? I have been assuming the ZK system would be fully integrated into the drafting and editing phase, especially because Zettlr emphasizes its ability to seamlessly insert Zotero citations. Do you outline the whole paper in Zettels and then expand it elsewhere? How do you deal with changes in paper structure that occur to you while writing?

    I have modified the output by Zotfile in the deep settings (e.g. the fundamentals in Zoteros core), so that I can simply copy and paste them and have Zettlr automatically convert it to Markdown. With the corresponding links, I can then quickly switch back to the PDF, if I need more context information.

    I'm not quite following. After you extract annotations in Zotero with Zotfile, you then copy/paste them into Zettlr and link/tag/divide them into multiple zettels? And your method retains the Zotero links to the pdf location?

    Nevertheless, what methodology are you talking about that Zotfile has that does not pertain reference management?

    I'm talking about using italics to distinguish between pdf quotes and my comments and automatically putting in links to the quote location in the pdf.

    However, since 1.5, Zettlr should tell you exactly which key was wrong, if it stopped loading your library. It's most probable something like this. Keep an eye out for the notifications in the top-right corner of the main window ;)

    I would really appreciate if you could give me more specific troubleshooting steps in the other thread about this. I did try 1.6 as you suggested. Do I need to delete the json database and re-import it?

    Re: keyboard shortcuts, I totally understand your time limitations, and I will propose features/shortcuts on GitHub.

  • Maybe it helps to outline my full workflow here, so that you can see how I integrate the Zettelkasten into my workflow!

    First: Read a PDF, annotating as I go.

    Second: Make sure in the Config Editor that the format for the extracted annotations fonforms with the way I format things (highlighted sections --> Blockquote; My own stuff in a simple paragraph <p>)

    Then: Extract the annotations

    Arrive at a super simple layouted RTF document:

    Then select all text and copy it, create a new note in Zettlr and simply paste it. The HTML-to-Markdown-converter will make sure the annotations are rendered perfectly. Note that I tend to separate different chapters of books into one directory with one-file-per-chapter logic. This "pasted text" was only to demonstrate how it looks like.

    Then I use tags to immediately identify the status of one of these massive excerpts (each one has between 1,000 and 5,000 words)

    Afterwards, I re-read the annotations, and always add at least one paragraph to each block quote from the text describing its contents for myself in simple words:

    Then I mostly use these excerpts, because I mostly don't have time for the following points, but I'll include them for completeness:

    After the files have been correctly worked on and are finished, I have a handy python script that looks for these headers level 4, which I exclusively use for my very own thought-titles. It splits up the excerpt files into individual notes and performs other black magic under the hood. What it does is:

    Split up ALL files into their atomic notes (according to header level 4), create outlines for each chapter and each book, add the literature reference to the final notes, link everything back and forth and create a Zettelkasten simply from all my long excerpts:


    Finally, when I write, I actually see Zettlr as a two-part program: To the right is the document I am working on, while on the left are my work notes. I use the global search to find things, and then ALT-Click the notes to open them in QuickLook Windows.


    So, and now to the other questions :smiley:

    Do I need to delete the json database and re-import it?

    I don't think that you need to delete it. Zettlr tries to parse it as soon as it has "changed" (that is: The library file in the preferences has changed), and then it will output you everything that has gone wrong. To trigger this re-load, simply empty out the library field in the preferences (Ctrl+A and then Ctrl+X), save the preferences, then re-open the preferences, and Ctrl+V the path back in. Hit save again and trigger the reload. Then, any errors should be displayed in the top-right corner!

  • Wow, thank you for taking so much time to describe your process with screenshots! Here are some questions:

    • You tag your annotations/long notes before breaking them into atomic notes, correct? Does your Python program duplicate these tags at the top of each atomic note?
    • You say you use tags to identify workflow status. Do you also use them to identify content or topic, or just for workflow organization purposes? Figuring out the right way to use tags for topics is one of my current challenges.

    Here are the steps in your writing workflow I am still not clear about:

    • When you are working with a large number of atomic notes from different sources, how are you using Zettlr to form new connections and ideas between them, and what do the Zettels capturing these new ideas look like? What role (if any) do tags play in this process?
    • Once you have an idea of your argument, how are you organizing these Zettel into an outline? Could you show an example Zettel that is a chapter outline and simply links to other Zettels? The thing I am not understanding is that the atomic notes are individual ideas, but they are written in a way that is ideal for your future understanding, not in a way that allows them to easily communicate to others nor in a way that allows them to seamlessly flow from one to the other in the style of a written article. Phrases such as "Since this strategy cannot be defended, we must consider others" don't seem to belong on a Zettel but may be essential to a paper.
  • You tag your annotations/long notes before breaking them into atomic notes, correct? Does your Python program duplicate these tags at the top of each atomic note?

    No, my Python program doesn't do that currently; at the current time it's more some kind of a line-by-line reader which is pretty dumb. I need to rewrite that thing to make use of ASTs and stuff, when I have time :D

    You say you use tags to identify workflow status. Do you also use them to identify content or topic, or just for workflow organization purposes? Figuring out the right way to use tags for topics is one of my current challenges.

    I basically have two sets of tags I make use of. One are kind of meta-tags I solely use to colour my notes in the file list, that is: #todo, #done, #formatting, #review, etc. The second set are the "real" tags I use to categorise my notes with, which I did not assign a colour in the tag manager, hence I simply drop them in where I deem them fit. I don't have any strategy for using them, because I simply hope to be using the same tags for the same topic, but I don't care too much!

    When you are working with a large number of atomic notes from different sources, how are you using Zettlr to form new connections and ideas between them, and what do the Zettels capturing these new ideas look like? What role (if any) do tags play in this process?

    Well, only if I realise "Wait, there's this and that note that fits!", so currently I'm not so much caring for my Zettelkasten due to time constraints. But also, because I want to implement some form of natural language processing to Zettlr to have the app automatically suggest some notes for me, it's all about being lazy and then starting huge projects to support my laziness :D

    Once you have an idea of your argument, how are you organizing these Zettel into an outline?

    I don't actually. My writing flow is pretty much about producing one consistent stream of text. I am doing pretty bad with creating an outline and then expanding it to a text; so I'm normally having my notes at the side of my text and use/cite them as they fit in. Normally I write a sentence, where I want to say some broad concept of what I have been reading, and then I pull up notes pertaining that. Sometimes, this makes me remember further information which I simply drop in there, but otherwise I'll simply look where I took that thought from and then drop in the correct citation!

    Could you show an example Zettel that is a chapter outline and simply links to other Zettels?

    Here you go:

    The thing I am not understanding is that the atomic notes are individual ideas, but they are written in a way that is ideal for your future understanding, not in a way that allows them to easily communicate to others nor in a way that allows them to seamlessly flow from one to the other in the style of a written article. Phrases such as "Since this strategy cannot be defended, we must consider others" don't seem to belong on a Zettel but may be essential to a paper.

    Well, yes. I mean, after all what you write on your notes is highly a matter of subjective needs. As my workflow does not directly start at Zettels but at Excerpts (because I sometimes need the full argumentational timeline from the text I was reading to contextualise some things), they are not as "individual" as you might think.

    Overall, the full concept of a Zettelkasten is still very much vaguely defined, so it's difficult to pinpoint do's and dont's …

  • Wow, this was tremendously insightful! Thank you @hendrik for sharing these details of your workflow! Even though Zettlr has a fairly good documentation, the documentation only explains how Zettlr works, not how you are supposed to work with it. Well, I guess "supposed to" is too strong, I mean potential ways of working with it. This is not a criticism, it is basically true for any software. But after all, it makes a difference whether you just read a recipe in a cookbook, or if you are in the kitchen with the chef and watch their process.

  • the documentation only explains how Zettlr works, not how you are supposed to work with it.

    Precisely! But this is why I tend to separate my workflow from the documentation, because, after all, it's just my own workflow, not one suited for everyone — and besides, I tend to see my workflow as fundamentally flawed in many respects, because of the issues mentioned! But I'm glad it helped :)

  • Yes, thanks for the information. I just read How to Take Smart Notes and I am starting to see that the ZK method is actually less well-defined in some senses than I thought. Things like tagging and connecting notes seem to be very individual, so maybe it will get clearer for me only once I start using it more.

  • Yes, absolutely! If you stumble upon good methods, please feel free to share them :)

  • it's all about being lazy and then starting huge projects to support my laziness

    :)
    This is super cool. I really liked this and now I want a python script too.

  • I know this is an old discussion, but I have really been struggling with how to implement the zettelkasten method and I think this explanation of your workflow has finally helped bridge the gap so I'm ready to start... Hopefully... Thank you for being willing to explain your workflow in such detail (and thank you for this great software)!

  • Hendrik,

    Thank you for the very helpful information. In the entire ZK ecosystem, I think actual examples of implementations are the single-most important rare items.

    I do have two questions:

    1. Especially when reading a complex, long document, I want to read, annotate, and then zettelize individual chapters, rather than wait until I finish reading the entire book. Many great books can take weeks to read or sections are best read on an as-needed basis. Also, there are edited collections of articles. Do you have a way to your #4 step piecemeal, rather than wait until you've read and annotated the entire document?

    2. Right now my NAS drive is in the repair shop, so I don't have ready access to my pdf collection. I can copy an annotated pdf to my workstation's hard drive, attach it to Zotero, and then extract the annotations with Zotfile. But ordinarily the annotated pdf would go to the NAS drive. I don't want to start saving linked pdf's in multiple locations, because I'm afraid I won't be able to keep track of them. Do you have a way to go straight from the annotated pdf to Zettlr without having to first save it in your Zotero library?

Sign In or Register to comment.