encoding/xml: Decoder does not handle external entities correctly

[gopherbot]

by yogischogi:

What steps will reproduce the problem?
If possible, include a link to a program on play.golang.org.

Decode a DocBook file containing external entities
http://play.golang.org/p/-szb21L82T

What is the expected output?

Entity names with a trailing ";"


What do you see instead?

1. When Decoder.Strict = true 
   the decoder reports an error
   "Invalid character entity".

2. When Decoder.Strict = false
   in most cases, the trailing ";" is not appended.
   When the entity contains "-" the ";" is appended.


Which version are you using?  (run 'go version')

go1.0.3

[rsc]

Comment 1:

Labels changed: added priority-later, go1.1, removed priority-triage.

Status changed to Accepted.

[rsc]

Comment 3:

Labels changed: added size-m.

[rsc]

Comment 4:

Labels changed: added suggested.

[gopherbot]

Comment 5 by ymotongpoo:

I'm working on this.

[rsc]

Comment 6:

Fixed at tip, will be in Go 1.1.
Not sure when it was fixed, but http://play.golang.org/p/-szb21L82T works for me (see
below). I agree that the sandbox at Go 1.0.3 still has the bug.
g% cat x.go
package main
import (
    "encoding/xml"
    "fmt"
    "io"
    "strings"
)
func main() {
    docBook := `<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
    "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"
    [<!ENTITY preface SYSTEM "preface.xml">
     <!ENTITY town-and-country SYSTEM "town-and-country.xml">
]>
<article>
&preface;
&town-and-country;
</article>`
    decoder := xml.NewDecoder(strings.NewReader(docBook))
    decoder.Strict = false
    for {
        token, err := decoder.Token()
        if err != nil && err != io.EOF {
            fmt.Printf("Error! Decoding XML failed: %v\n", err)
            break
        }
        if token == nil {
            break
        }
        switch element := token.(type) {
        case xml.CharData:
            fmt.Printf("%s\n", element)
        }
    }
}
g% go run x.go
&preface;
&town-and-country;
g%

Status changed to Fixed.

[gopherbot]

Comment 7 by yogischogi:

Looks fine. Thanks to everyone who has been working on this.