commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raymond DeCampo <...@decampo.org>
Subject Re: [text] On the value of idempotent string escape methods?
Date Wed, 22 Feb 2017 12:41:12 GMT
On Mon, Feb 20, 2017 at 11:40 PM, Sampanna Kahu <sampyash@gmail.com> wrote:

> Hi Guys,
> Very good points are being made above. Please allow me to add my two cents
> :-)
>
> What if the string contains syntactically valid HTML characters/tags and
> our aim is to prevent rendering these tags in the browser when this string
> is being served via a web application? Or prevent the execution of harmful
> embedded scripts when serving it? The 'escapeOnce' method could be useful
> here, right?
>
> To explain better, let's consider an example of the specific use-case that
> I had in mind when building the 'escapeOnce' method:
> Consider the scenario of a simple restful web application where users can
> manipulate their text using simple crud operations. Lets assume that we do
> not have the 'escapeOnce' method yet.
> 1. A user comes and submits his string. We escape it and store it in our
> database. If the string had any HTML characters, they would have gotten
> escaped.
>

As others have pointed out, escaping the data and then storing it is a
mistake.


> 2. After some time, the same user fetches his string, adds some more HTML
> characters and submits it. At this point, although the escape method would
> correctly escape the freshly added HTML characters, it would escape the
> older escaped HTML characters again! (for example &gt; would become
> &amp;gt;)
>

Wouldn't you have unescaped the string before sending it to the user for
editing?


>
> And this effect gets magnified if step number 2 above is repeated.
>
> How do we solve the above problem without the 'escapeOnce' method?
>

I imagine that at this point you are stuck with the escaped data in the
database.  If that is the case, the best solution is to unescape the string
before allowing modifications.

If for some reason that is not an option, you can apply unescape to the
modified string and then apply escape.  This will prevent re-escaping of
existing escape sequences and will escape anything new that needs it.
Since this is just two consecutive calls to existing library functionality
it hardly seems burdensome on the users to omit a convenience function for
it.  (Especially since it encourages what are, IMHO, bad practices.)



>
> On 20 February 2017 at 21:40, sebb <sebbaz@gmail.com> wrote:
>
> > On 20 February 2017 at 15:36, Rob Tompkins <chtompki@apache.org> wrote:
> > >
> > >> On Feb 20, 2017, at 10:30 AM, sebb <sebbaz@gmail.com> wrote:
> > >>
> > >> On 20 February 2017 at 14:55, Rob Tompkins <chtompki@apache.org>
> wrote:
> > >>>
> > >>>> On Feb 20, 2017, at 4:31 AM, sebb <sebbaz@gmail.com> wrote:
> > >>>>
> > >>>> On 19 February 2017 at 14:29, Raymond DeCampo <ray@decampo.org
> > <mailto:ray@decampo.org>> wrote:
> > >>>>> I am trying to see how having the proposed unescape() method
leads
> > to an a
> > >>>>> useful escape method.
> > >>>>>
> > >>>>> E.g. clearly unescape("&amp;") would evaluate to "&".
 So would
> > >>>>> unescape("&amp;amp;").  That means the proposed escape()
method
> > would also
> > >>>>> have the same output for "&amp;" and "&amp;amp;".
> > >>>>>
> > >>>>> I think a better approach for an idempotent escape would be
to just
> > >>>>> unescape the string once, and then run the traditional escape.
> > >>>>
> > >>>> That does not eliminate the problems, as you state below.
> > >>>>
> > >>>>> You will
> > >>>>> still have issues if the user intended to escape the string
"&amp;"
> > but you
> > >>>>> are never going to crack that without some kind of state saving.
> > >>>>
> > >>>> That is my exact point.
> > >>>>
> > >>>> Since it's not possible for the function to work reliably, we should
> > >>>> not mislead users by pretending that there is a magic method that
> > >>>> works.
> > >>>>
> > >>>>> Than given that the functionality is available via to consecutive
> > calls to
> > >>>>> existing methods, I would probably be disinclined to include
it in
> > the
> > >>>>> library.
> > >>>>
> > >>>> +1
> > >>>
> > >>> I’m a (+1) for removal as well.
> > >>>
> > >>> Also, I didn’t mean for my example to sound like a proposal. I merely
> > was trying to get to a potentially valuable stateless idempotent string
> > escape function. Its contrivance it quite clear.
> > >>>
> > >>> Any other comments out there?
> > >>>
> > >>> We could provide a stateful escaper (that figures out how many
> escapes
> > a string is in), or a method that returns the number of escapes in a
> string
> > is. Again, I’m not all that sure on the value of such methods.
> > >>
> > >> I don't think it's possible to work out the number of times a string
> > >> has been escaped.
> > >
> > > That may indeed be true, but it is possible to return the number of
> > times unescape need be run before subsequent unescapes yield the same
> > result.
> >
> > That in itself is potentially ambiguous.
> > Does the unescaper keep going until there are no valid escape
> > sequences left, or does it stop when there is a least one ampersand
> > which is not part of a valid escape sequence?
> >
> > > Again, I’m not sure if this is a valuable measure to concern ourselves
> > with.
> >
> > I don't think it provides anything useful.
> >
> > >>
> > >> The most one can do is to determine if a string has not been escaped.
> > >> That would be the case where a string has one or more unescaped
> > >> characters in it.
> > >> For example "This & that" has obviously not been escaped.
> > >>
> > >> However if a string has no un-escaped characters it it, that does not
> > >> necessarily mean that it has already been escaped.
> > >> For example: "This &amp; that".
> > >> This might have been escaped - or it might not.
> > >
> > > Ah, I was using the definition of “having been escaped” to be that the
> > string contains escape sequences.
> > >
> > >> For example it could be the answer to: "How does one code 'This &
> > >> that' in HTML?”
> > >>
> > >> The application has to keep track of the escape-state of the string.
> > >
> > > Definitely agreed with your definition of “having been escaped."
> > >
> > >>
> > >>> Cheers,
> > >>> -Rob
> > >>>
> > >>>>
> > >>>>>
> > >>>>> On Sat, Feb 18, 2017 at 12:04 PM, Rob Tompkins <chtompki@gmail.com
> >
> > wrote:
> > >>>>>
> > >>>>>> In preparation for the 1.0 release, I think we should address
> Sebb's
> > >>>>>> concern in TEXT-40 about the attempt to create "idempotent"
string
> > escape
> > >>>>>> methods. By idempotent I mean someMethod("some string")
=
> > >>>>>> someMethod(someMethod(someMethod(...someMethod("some
> string")))), a
> > >>>>>> single application of a method is equal to any number of
the
> > applications
> > >>>>>> of the method on the same input.
> > >>>>>>
> > >>>>>> Below I lay out a mechanism by which it is possible to
write such
> > methods,
> > >>>>>> but I don’t know the value in writing such methods. I'm
merely
> > expressing
> > >>>>>> that idempotency is a possibility.
> > >>>>>>
> > >>>>>> For string "un-escaping", I believe that we can write a
method
> that,
> > >>>>>> indeed, is idempotent by simply running the un-escape method
the
> > finite
> > >>>>>> number of un-escapings to get to the point at which the
string
> > remains
> > >>>>>> unchanged between applications of the un-escaping method.
(I
> > believe that I
> > >>>>>> can write a proof that all un-escape methods have such
a point, if
> > that is
> > >>>>>> needed for the sake of discussion).
> > >>>>>>
> > >>>>>> If indeed we can create an idempotent un-escape method,
then we
> can
> > simply
> > >>>>>> take that method run it, and then run the escaping method
one
> time.
> > If we
> > >>>>>> always completely unescape and then escape once then we
do have an
> > >>>>>> idempotent method.
> > >>>>>>
> > >>>>>> Such a method might not be all that valuable to the user
though.
> > >>>>>> Furthermore, this just explains one way to create such
an
> idempotent
> > >>>>>> method. Whether or not more or more valuable methods exists,
would
> > take
> > >>>>>> some more though.
> > >>>>>>
> > >>>>>> Anyone have any thoughts? My feeling is that it might be
more
> > effort than
> > >>>>>> it's worth to ensure that any string is only "singly encoded.”
> > Further, we
> > >>>>>> probably should give a look at the “escape_once” methods
in
> > >>>>>> StringEsapeUtils.
> > >>>>>>
> > >>>>>> Cheers
> > >>>>>> -Rob
> > >>>>>> ------------------------------------------------------------
> > ---------
> > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >>>>>> For additional commands, e-mail: dev-help@commons.apache.org
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>> ------------------------------------------------------------
> ---------
> > >>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org <mailto:
> > dev-unsubscribe@commons.apache.org>
> > >>>> For additional commands, e-mail: dev-help@commons.apache.org
> <mailto:
> > dev-help@commons.apache.org>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >> For additional commands, e-mail: dev-help@commons.apache.org
> > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message