logging-log4j-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Remko Popma <remko.po...@gmail.com>
Subject Re: Make LogEvent implementations Externalizable
Date Fri, 11 Jul 2014 15:09:36 GMT
That is a nice reduction in size!

I also think the ExternalizedLayout idea is a very attractive option. That
way there is no pressure to include this in any particular release, we can
release it when we are confident that is ready. I also like the fact that
it does not replace the current serialization and it can be switched on and
off in configuration so if there is a bug, users can fall back to the
existing plain serialization.



On Fri, Jul 11, 2014 at 10:00 PM, Matt Sicker <boards@gmail.com> wrote:

> I would second the ExternalizedLayout. Layouts are the way to go for
> compatibility and are simpler. Particularly useful for alternative
> serialization protocols, too.
>
>
> On 11 July 2014 06:41, Gary Gregory <garydgregory@gmail.com> wrote:
>
> > I understand Ralph ' s concern but now is the time for this kind of
> > change.  Otherwise we will need even more clever solutions to get this
> kind
> > of size improvement. I'd love to see some performance numbers. The size
> > improvement is not negligible, which is great!
> >
> > Gary
> >
> >
> >
> > <div>-------- Original message --------</div><div>From: Ralph
Goers <
> > ralph.goers@dslextreme.com> </div><div>Date:07/11/2014  01:46
> >  (GMT-05:00) </div><div>To: Log4J Users List <
> > log4j-user@logging.apache.org> </div><div>Subject: Re: Make LogEvent
> > implementations Externalizable </div><div>
> > </div>I’d be afraid of breaking compatibility even now.  However, I think
> > what you really want to do is to create an ExternalizedLayout and then
> just
> > use that instead of the default SerializedLayout.  If you want to supply
> > that Layout as a patch to a Jira issue it could be added at any time.
> >
> > Ralph
> >
> > On Jul 10, 2014, at 9:23 PM, Scott Harrington <scotth01@sns-usa.com>
> > wrote:
> >
> > > Ralph & co:
> > >
> > > I hear you're gearing up for the release.
> > >
> > > Last weekend I scratched an itch of mine relating to SocketAppender ->
> > SocketServer bandwidth, and was able to reduce a 500-character message
> from
> > around 1700 bytes to 700 bytes on the wire (it's easy to improve on
> Java's
> > default serialization).
> > >
> > > I was going to submit an enhancement request with patch to JIRA but
> > instead I went on vacation for two weeks.
> > >
> > > I made RingBufferLogEvent implement Externalizable, i.e. hand-coded
> > writeExternal / readExternal methods. I did NOT have time to make an
> > equivalent change to Log4jLogEvent, or to write up any performance tests
> or
> > regression tests.
> > >
> > > Should I submit what I have for discussion and hopeful inclusion in
> 2.0?
> > >
> > > Or will it have to wait for 2.1?
> > >
> > > If we wait, then due to the necessary serialVersionUID change, v2.0
> > SocketAppender would not be able to talk to v2.1 SocketServer or vice
> versa
> > (unless ugly duplicate versions are maintained).
> > >
> > > Below is what the added code looks like. I only tested in
> > RingBufferLogEvent but should be similarly usable in Log4jLogEvent, and
> > perhaps we should discuss a RingBufferLogEvent.readResolve that makes
> them
> > all become Log4jLogEvents on the SocketServer (receiving) end.
> > >
> > > ...
> > >
> > >    public void writeExternal(ObjectOutput out) throws IOException {
> > >        getThrownProxy();
> > >        out.writeByte(1); // wireFormat
> > >        int presenceMap = (loggerName == null ? 0 : 0x1) |
> > >            (marker == null ? 0 : 0x2) |
> > >            (fqcn == null ? 0 : 0x4) |
> > >            (level == null ? 0 : 0x8) |
> > >            (message == null ? 0 : (0x10 | (isSerializeAsString(message)
> > ? 0 : 0x20))) |
> > >            (thrownProxy == null ? 0 : 0x40) |
> > >            (contextMap == null ? 0 : 0x80) |
> > >            (contextStack == null ? 0 : 0x100 | (contextStack.getDepth()
> > == 0 ? 0 : 0x200)) |
> > >            (threadName == null ? 0 : 0x400) |
> > >            (location == null ? 0 : 0x800);
> > >        out.writeShort(presenceMap);
> > >        if (loggerName != null) {
> > >            out.writeUTF(loggerName);
> > >        }
> > >        if (marker != null) {
> > >            out.writeObject(marker);
> > >        }
> > >        if (fqcn != null) {
> > >            out.writeUTF(fqcn);
> > >        }
> > >        if (level != null) {
> > >            out.writeUTF(level.name());
> > >        }
> > >        if (message != null) {
> > >            if (isSerializeAsString(message)) {
> > >                out.writeUTF(message.getFormattedMessage());
> > >            }
> > >            else {
> > >                out.writeObject(message);
> > >            }
> > >        }
> > >        if (thrownProxy != null) {
> > >            out.writeObject(thrownProxy);
> > >        }
> > >        if (contextMap != null) {
> > >            writeString2StringMap(out, contextMap);
> > >        }
> > >        if (contextStack != null && contextStack.getDepth() != 0) {
> > >            out.writeObject(contextStack);
> > >        }
> > >        if (threadName != null) {
> > >            out.writeUTF(threadName);
> > >        }
> > >        if (location != null) {
> > >            out.writeUTF(location.getClassName());
> > >            out.writeUTF(location.getMethodName());
> > >            if ((presenceMap & 0x1000) != 0) {
> > >                out.writeUTF(location.getFileName());
> > >            }
> > >            out.writeInt(location.getLineNumber());
> > >        }
> > >        out.writeLong(currentTimeMillis);
> > >        out.writeBoolean(endOfBatch);
> > >        out.writeBoolean(includeLocation);
> > >    }
> > >
> > >    public void readExternal(ObjectInput in) throws IOException,
> > ClassNotFoundException {
> > >        int wireFormat = in.readByte();
> > >        if (wireFormat == 1) {
> > >            int presenceMap = in.readShort();
> > >            loggerName = (presenceMap & 0x1) == 0 ? null :
> > >                in.readUTF();
> > >            marker = (presenceMap & 0x2) == 0 ? null :
> > >                (Marker) in.readObject();
> > >            fqcn = (presenceMap & 0x4) == 0 ? null :
> > >                in.readUTF();
> > >            level = (presenceMap & 0x8) == 0 ? null :
> > >                Level.valueOf(in.readUTF());
> > >            message = (presenceMap & 0x10) == 0 ? null :
> > >                (presenceMap & 0x20) == 0 ? new
> > SimpleMessage(in.readUTF()) : (Message) in.readObject();
> > >            thrownProxy = (presenceMap & 0x40) == 0 ? null :
> > >                (ThrowableProxy) in.readObject();
> > >            contextMap = (presenceMap & 0x80) == 0 ? null :
> > >                readString2StringMap(in);
> > >            contextStack = (presenceMap & 0x100) == 0 ? null :
> > >                (presenceMap & 0x200) == 0 ? ThreadContext.EMPTY_STACK :
> > (ContextStack) in.readObject();
> > >            threadName = (presenceMap & 0x400) == 0 ? null :
> > >                in.readUTF();
> > >            location = (presenceMap & 0x800) == 0 ? null :
> > >                new StackTraceElement(in.readUTF(), in.readUTF(),
> > (presenceMap & 0x1000) == 0 ? null : in.readUTF(), in.readInt());
> > >            currentTimeMillis = in.readLong();
> > >            endOfBatch = in.readBoolean();
> > >            includeLocation = in.readBoolean();
> > >        }
> > >        else {
> > >            throw new StreamCorruptedException("Unsupported LogEvent
> wire
> > format " + wireFormat);
> > >        }
> > >    }
> > >
> > >    private static boolean isSerializeAsString(Message message)
> > >    {
> > >        return message instanceof SimpleMessage || message instanceof
> > ObjectMessage;
> > >    }
> > >
> > >    private void writeString2StringMap(ObjectOutput out, Map<String,
> > String> map) throws IOException
> > >    {
> > >        out.writeInt(map.size());
> > >        for (Map.Entry<String, String> entry : map.entrySet()) {
> > >            out.writeUTF(entry.getKey());
> > >            out.writeUTF(entry.getValue());
> > >        }
> > >    }
> > >
> > >    private static Map<String, String> readString2StringMap(ObjectInput
> > in) throws ClassNotFoundException, IOException
> > >    {
> > >        int size = in.readInt();
> > >        if (size == 0) {
> > >            return Collections.emptyMap();
> > >        }
> > >        Map<String, String> map = new HashMap<String, String>(size);
> > >        while (size-- > 0) {
> > >            map.put(in.readUTF(), in.readUTF());
> > >        }
> > >        return map;
> > >    }
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > > For additional commands, e-mail: log4j-user-help@logging.apache.org
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: log4j-user-unsubscribe@logging.apache.org
> > For additional commands, e-mail: log4j-user-help@logging.apache.org
> >
> >
>
>
> --
> Matt Sicker <boards@gmail.com>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message