Improve BibTeX parsing and bibliography rendering

scribble-doc/scriblib/scribblings/autobib.scrbl

@@ -50,6 +50,15 @@ includes a citation to section 8 of the Racket reference.
 
 @history[#:changed "1.61"
   @elem{Added fields and location types for better bibtex support.}]
+@history[#:changed "1.68"
+  @elem{Improved bibliography layout, added support for
+        multi-paragraph notes and extended support for
+        structured content in bibliography fields.
+        Location helpers now return Scribble content,
+        potentially @racket[#f], rather than always an element.
+        Existing clients must not assume that
+        @racket[element-content] applies to their results.}]
+
 
 @defform/subs[(define-cite ~cite-id citet-id generate-bibliography-id
                            option ...)
@@ -158,11 +167,44 @@ optionally given @racket[render-date-expr] functions.
 
 Styles for use with @racket[define-cite].
 
+With @racket[number-style], bibliography entries use hanging
+indentation in HTML and LaTeX output, with citation numbers
+aligned in a separate label column. Text output instead
+separates each number from its entry with a non-breaking space.
+
 The @racket[author+date-square-bracket-style] definition is the same
 as @racket[author+date-style], except that references to citations
 are enclosed in @litchar["[]"] instead of @litchar["()"].
-}
 
+In LaTeX output, Scribble tries to keep short bibliography
+entries together, reserving at least five lines before starting
+an entry. This approximates the previous behavior, which
+prevented page breaks within individual entries altogether.
+Longer entries may now span pages.
+
+The @tt{\AutobibNeedlines} counter controls the minimum number of lines,
+defaulting to 5. Set it to 0 to disable this constraint.
+The optional @tt{needspace} package is required
+for the constraint to take effect;
+this behavior is disabled if the package is unavailable,
+as if the counter were 0.
+
+The @tt{\AutobibEntrySetup} command, empty by default,
+allows additional LaTeX settings to be applied locally
+to each bibliography entry.
+
+To require four lines before each entry and relax line breaking
+for long annotations, configure these settings using @tt{\AtBeginDocument}
+from e.g. a @racket[tex-addition] that you add to your document's style:
+
+@racketblock[
+(tex-addition
+  (bytes-append
+    #"\\AtBeginDocument{%\n"
+    #"  \\AutobibNeedlines=4\\relax\n"
+    #"  \\renewcommand{\\AutobibEntrySetup}{%\n"
+    #"    \\emergencystretch=2em\n"
+    #"    \\tolerance=1000}}%\n"))]}
 
 @defproc[(bib? [v any/c]) boolean?]{
 
@@ -176,6 +218,7 @@ Returns @racket[#t] if @racket[v] is a value produced by
                    [#:location location any/c #f]
                    [#:date date (or/c #f date? exact-nonnegative-integer? string?) #f]
                    [#:url url (or/c #f string?) #f]
+                   [#:accessed accessed any/c #f]
                    [#:doi doi (or/c #f string?) #f]
                    [#:note note any/c #f])
          bib?]{
@@ -187,6 +230,23 @@ supplied. Functions like @racket[proceedings-location],
 @racket[author-name], and @racket[authors] help produce elements in a
 standard format.
 
+The @racket[#:note] argument may contain multiple paragraphs,
+separated by blank lines. The first paragraph follows the
+bibliographic information; subsequent paragraphs remain within
+the same bibliography entry.
+
+When both @racket[#:doi] and @racket[#:url] are supplied,
+the DOI takes precedence.
+A period is inserted after a DOI when followed by a non-empty note.
+No period is appended directly to a URL.
+
+@racket[#:accessed] gives the date a @racket[#:url] (in CSL terms, the
+date it was accessed), and is only used when @racket[#:url] is displayed,
+i.e. when no @racket[#:doi] is supplied. It is rendered right after the
+URL, separated from it by a space, as @tt{(accessed ...)}. A period is
+inserted after it when followed by a non-empty note; a naked URL (no
+@racket[#:accessed]) never gets that period.
+
 Dates are internally represented as @racket[date] values, so a @racket[date]
 may be given, or a number or string that represent the year.
 
@@ -197,7 +257,13 @@ name, the last non-empty sequence of alphabetic characters or
 @litchar["-"] after a space is treated as the author name, and the
 rest is treated as the first name.
 
-@history[#:changed "1.49" @elem{Added @racket[#:doi].}]}
+@history[#:changed "1.49" @elem{Added @racket[#:doi].}]
+@history[#:changed "1.68"
+  @elem{Added @racket[#:accessed], which replaces the accessed-date
+        support formerly provided by the now-removed @tt{webpage-location}
+        function: the accessed date is now attached directly to the
+        bib entry instead of being embedded in its @racket[#:location],
+        so it renders next to the URL rather than before the date.}]}
 
 @defproc[(in-bib [orig bib?] [where string?]) bib?]{
 
@@ -213,24 +279,32 @@ Extends a bib value so that the rendered citation is suffixed with
                                [#:organization organization any/c #f]
                                [#:publisher publisher #f]
                                [#:address address #f])
-         element?]{
+         (or/c content? #f)]{
 
-Combines elements to generate an element that is suitable for
+Combines the supplied information to produce content suitable for
 describing a paper's location within a conference or workshop
 proceedings.
 
- @history[#:changed "1.61"
-   @elem{Added fields for bibtex support: editor number organization publisher address.}]
+@history[#:changed "1.61"
+  @elem{Added fields for bibtex support: editor number organization publisher address.}]
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, potentially
+        @racket[#f], rather than necessarily an element.}]
 }
 
 @defproc[(journal-location [title any/c]
                            [#:volume volume any/c #f]
                            [#:number number any/c #f]
                            [#:pages pages (or (list/c any/c any/c) #f) #f])
-         element?]{
+         (or/c content? #f)]{
 
-Combines elements to generate an element that is suitable for
-describing a paper's location within a journal.}
+Combines the supplied information to produce content suitable for
+describing a paper's location within a journal.
+
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, potentially
+        @racket[#f], rather than necessarily an element.}]
+}
 
 
 @defproc[(book-location [#:edition edition any/c #f]
@@ -242,78 +316,86 @@ describing a paper's location within a journal.}
                         [#:pages pages any/c #f]
                         [#:publisher publisher any/c #f]
                         [#:address address any/c #f])
-         element?]{
-Combines elements to generate an element that is suitable for
+         (or/c content? #f)]{
+Combines the supplied information to produce content suitable for
 describing a book's location.
 
+A numeric @racket[chapter], supplied as a number or a string
+of decimal digits, is prefixed with ``chapter''.
+Other chapter content is used unchanged.
+
 @history[#:changed "1.61"
   @elem{Added fields for bibtex support: editor chapter series volume number pages address.
         Made all arguments optional.}]
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, potentially
+        @racket[#f], rather than necessarily an element.}]
 }
 
 
 @defproc[(booklet-location [#:howpublished howpublished any/c #f]
                            [#:address address any/c #f])
-         element?]{
-Combines elements to generate an element that is suitable for
+         (or/c content? #f)]{
+Combines the supplied information to produce content suitable for
 describing a booklet's location.
 
 @history[#:added "1.61"]
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, potentially
+        @racket[#f], rather than necessarily an element.}]
 }
 
 
 @defproc[(misc-location [#:howpublished howpublished any/c #f])
-         element?]{
-Combines elements to generate an element that is suitable for
+         (or/c content? #f)]{
+Combines the supplied information to produce content suitable for
 describing a bibtex misc entry's location.
-
 @history[#:added "1.61"]
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, potentially
+        @racket[#f], rather than necessarily an element.}]
 }
 
 
 @defproc[(manual-location [#:organization organization any/c #f]
                           [#:edition edition any/c #f])
-         element?]{
-Combines elements to generate an element that is suitable for
+         (or/c content? #f)]{
+Combines the supplied information to produce content suitable for
 describing a manual's location.
 
 @history[#:added "1.61"]
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, potentially
+        @racket[#f], rather than necessarily an element.}]
 }
 
 
 @defproc[(techrpt-location [#:institution institution any/c]
                            [#:type type any/c #f]
                            [#:number number any/c #f]
                            [#:address address any/c #f])
-         element?]{
-
-Combines elements to generate an element that is suitable for
+         (or/c content? #f)]{
+Combines the supplied information to produce content suitable for
 describing a technical report's location.
 
 @history[#:changed "1.61" @elem{Added fields for bibtex support: type address.}]
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, potentially
+        @racket[#f], rather than necessarily an element.}]
 }
 
 @defproc[(dissertation-location [#:institution institution any/c]
                                 [#:degree degree any/c "PhD"]
                                 [#:type type any/c #f]
                                 [#:address address any/c #f])
-         element?]{
-
-Combines elements to generate an element that is suitable for
+         content?]{
+Combines the supplied information to produce content suitable for
 describing a dissertation.
 
 @history[#:changed "1.61"
   @elem{Added fields for bibtex support: type address.}]
-}
-
-@defproc[(webpage-location [url string? #f]
-                           [#:accessed accessed any/c #f])
-         element?]{
- Combines elements to generate an element that is suitable for
- describing a web page.
-
- @history[#:changed "1.61"
-   @elem{Made field url optional now that any autobib entry may have a url.}]
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, rather than necessarily an element.}]
 }
 
 
@@ -327,13 +409,17 @@ describing a dissertation.
                                 [#:pages pages any/c #f]
                                 [#:publisher publisher any/c #f]
                                 [#:address address any/c #f])
-         element?]{
-
-Combines elements to generate an element that is suitable for
+         (or/c content? #f)]{
+Combines the supplied information to produce content suitable for
 describing a paper's location within a chapter or part of a book or collection of books.
 
+The @racket[chapter] argument is formatted as by @racket[book-location].
+
 @history[#:changed "1.61"
   @elem{Added fields for bibtex support: editor chapter number address.}]
+@history[#:changed "1.68"
+  @elem{Now returns Scribble content, potentially
+        @racket[#f], rather than necessarily an element.}]
 }
 
 

scribble-doc/scriblib/scribblings/bibtex.scrbl

@@ -19,17 +19,31 @@ We support the 14 BibTeX entry types documented
 @litchar{manual}, @litchar{mastersthesis}, @litchar{misc}, @litchar{phdthesis},
 @litchar{proceedings}, @litchar{techreport}, and @litchar{unpublished}.
 
+Human-readable BibTeX fields are converted from a subset of
+LaTeX syntax into Scribble content. This includes grouping
+braces, common accent and special-letter commands,
+@tt{\emph}, @tt{\texttt}, @tt{\textit}, @tt{\textbf},
+@tt{\textsc} and @tt{\url}.
+
+Grouping braces are preserved internally where relevant
+to BibTeX name parsing. Unknown LaTeX commands and their
+attached arguments are retained rather than discarded.
+Inline and display mathematics are preserved in LaTeX
+syntax; other backends do not translate them into native
+mathematical expressions.
+
+Blank lines in @litchar{note} fields separate paragraphs.
+
+The @litchar{url} and @litchar{doi} fields are interpreted
+as scalar strings rather than general LaTeX content.
+Enclosing brace groups are removed and conventional
+LaTeX escapes for special URL characters, such as
+@litchar{\_}, @litchar{\%} and @litchar{\&}, are unescaped.
+Other LaTeX commands are not interpreted in these fields.
+
 We support all the required and optional fields documented in the LaTeX book,
 with the following known limitations so far:
 @itemize[
-  @item{We enclose the string parsed from field @litchar{title} in
-         @racket["(elem #:style (make-style #f '(exact-chars)) title)"],
-         which will directly include its text in the output.
-         This will presumably do the Right Thing™ when using the LaTeX backend,
-         but the wrong thing in the HTML backend.}
-  @item{Other fields are just parsed as strings,
-         and may appear as source code rather than as formatted code
-         in both the LaTeX and HTML backends.}
   @item{We fail to process @litchar{month}.}
   @item{We only support @litchar{pages} fields that have decimal numbers
         separated by one or more dashes.}
@@ -56,6 +70,10 @@ We do support the @litchar["@string"] feature defined in
   @elem{Support all standard entry types plus @litchar{online} and @litchar{webpage},
   all fields but @litchar{month} (or @litchar{type} for @litchar{incollection}),
   and support @litchar{note}, @litchar{url}, @litchar{doi} on all entry types.}]
+@history[#:changed "1.68"
+  @elem{Added structured LaTeX content parsing, improved
+        author-name handling, URL and DOI unescaping,
+        and support for multi-paragraph notes.}]
 
 @defform[(define-bibtex-cite bib-pth ~cite-id citet-id generate-bibliography-id
            option ...)]{

scribble-lib/info.rkt

@@ -21,7 +21,7 @@
 
 (define pkg-authors '(mflatt eli))
 
-(define version "1.67")
+(define version "1.68")
 
 (define license
   '((Apache-2.0 OR MIT)