INN commit: branches/2.6 (16 files)

INN Commit rra at isc.org
Mon May 14 12:42:14 UTC 2018


    Date: Monday, May 14, 2018 @ 05:42:14
  Author: iulius
Revision: 10282


Add support for embedded Python 3 interpreter to use with innd and nnrpd filter hooks

* Update configure script to find Python 3 interpreter and correctly
set linker flags for embedding it.  Python 3.3.0 or later in the 3.x
series is now supported.

* Drop support for Python 2.2.0; now, Python 2.3.0 or later in the 2.x
series is required  because configure looks for a shared Python library,
installed in the main library location in Python 2.3.0 and later.

* Add m4 macros to check for minimal Python version and module presence
at configure time.

* Notable changes for Python 3 are:
- string literals are now considered as Unicode data whereas they were
mere bytes in Python 2.  Consequently, encoding now really matters for
strings, and UTF-8 is required in return values of Python filter hooks.
(Note that compliance with NNTP would also want strings to be encoded
in UTF-8.)  Care should be given to use the right str or bytes objects
in Python 3.

- buffer objects no longer exist:  they have been replaced with
memoryview objects.  Consequently, code to deal with them slighty
changed.

- integers no longer exist in the C API:  they have been replaced with
long integers (Py_ssize_t).

- embedded Python initialization is now done differently (with a
PyMODINIT_FUNC function).  Try to homogeneize initialization with
Python 2.x.

* Improve error/notice logs.

* Update and improve both documentation and samples accordingly.

* Remove unused PYpathkey char * variable.

* A few typo fixes.

Modified:
  branches/2.6/configure.ac
  branches/2.6/doc/pod/hook-python.pod
  branches/2.6/doc/pod/install.pod
  branches/2.6/doc/pod/news.pod
  branches/2.6/innd/innd.h
  branches/2.6/innd/python.c
  branches/2.6/m4/python.m4
  branches/2.6/nnrpd/nnrpd.h
  branches/2.6/nnrpd/python.c
  branches/2.6/samples/filter_innd.py
  branches/2.6/samples/nnrpd_access.py
  branches/2.6/samples/nnrpd_access_wrapper.py
  branches/2.6/samples/nnrpd_auth.py
  branches/2.6/samples/nnrpd_auth_wrapper.py
  branches/2.6/samples/nnrpd_dynamic.py
  branches/2.6/samples/nnrpd_dynamic_wrapper.py

----------------------------------+
 configure.ac                     |   23 +++++++
 doc/pod/hook-python.pod          |   90 ++++++++++++++++--------------
 doc/pod/install.pod              |   11 ++-
 doc/pod/news.pod                 |   27 +++++++++
 innd/innd.h                      |    5 +
 innd/python.c                    |   62 +++++++++++++++++---
 m4/python.m4                     |  111 ++++++++++++++++++++++++++-----------
 nnrpd/nnrpd.h                    |    5 +
 nnrpd/python.c                   |   98 ++++++++++++++++++++++++--------
 samples/filter_innd.py           |   51 ++++++++++-------
 samples/nnrpd_access.py          |   43 ++++++++------
 samples/nnrpd_access_wrapper.py  |   27 +++++----
 samples/nnrpd_auth.py            |   49 ++++++++++------
 samples/nnrpd_auth_wrapper.py    |   23 ++++---
 samples/nnrpd_dynamic.py         |   59 +++++++++++--------
 samples/nnrpd_dynamic_wrapper.py |   11 ++-
 16 files changed, 475 insertions(+), 220 deletions(-)

Modified: configure.ac
===================================================================
--- configure.ac	2018-05-14 12:33:11 UTC (rev 10281)
+++ configure.ac	2018-05-14 12:42:14 UTC (rev 10282)
@@ -223,9 +223,28 @@
      AC_SUBST([PERL_LIBS])
      AC_SUBST([PERL_WARNINGS])])
 
-dnl Check for embedded Python interpreter.
-INN_ARG_PYTHON
+dnl Support for embedded Python.
+AC_ARG_WITH([python],
+    [AS_HELP_STRING([--with-python],
+                    [Embedded Python module support @<:@no@:>@])],
+    [AS_CASE([$withval],
+        [yes],
+            [DO_PYTHON=DO
+             AC_DEFINE([DO_PYTHON], [1],
+                 [Define to compile in Python module support.])],
+        [no],
+            [DO_PYTHON=DONT],
+        [AC_MSG_ERROR([invalid argument to --with-python])])],
+    [DO_PYTHON=DONT])
 
+dnl Checks for a recent enough Python interpreter for embedded Python.
+dnl Requires 2.3.0 because configure looks for a shared Python library,
+dnl installed in the main library location in Python 2.3.0 and later.
+dnl Python 3.3.0 and later is also supported.
+AS_IF([test x"$DO_PYTHON" = xDO],
+    [INN_PROG_PYTHON([2.3.0], [3.3.0])
+     INN_LIB_PYTHON])
+
 dnl Set some configuration file defaults from the machine hostname.
 HOSTNAME=`hostname 2> /dev/null || uname -n`
 AC_SUBST(HOSTNAME)

Modified: doc/pod/hook-python.pod
===================================================================
--- doc/pod/hook-python.pod	2018-05-14 12:33:11 UTC (rev 10281)
+++ doc/pod/hook-python.pod	2018-05-14 12:42:14 UTC (rev 10282)
@@ -4,10 +4,9 @@
 filtering.  It is patterned after the Perl and (now obsolete) TCL hooks
 previously added by Bob Heiney and Christophe Wolfhugel.
 
-For this filter to work successfully, you will need to have
-at least S<Python 2.2.0> installed.  You can obtain it from
-L<http://www.python.org/>.  Please note that S<Python 3.x> is currently
-not supported.
+For this filter to work successfully, you will need to have at least
+S<Python 2.3.0> (in the 2.x series) or S<Python 3.3.0> (in the 3.x
+series) installed.  You can obtain it from L<http://www.python.org/>.
 
 The B<innd> Python interface and the original Python filtering documentation
 were written by Greg Andruk (nee Fluffy) <gerglery at usa.net>.  The Python
@@ -93,14 +92,16 @@
 by your INN (especially, the Xref: header, if present, is the one
 of the remote site which sent you the article, and not yours).
 
-These values will be buffer objects holding the contents of the
-same named article headers, except for the special C<__BODY__> and C<__LINES__>
+These values will be buffer objects (for S<Python 2.x>) or memoryview
+objects (for S<Python 3.x>) holding the contents of the same named
+article headers, except for the special C<__BODY__> and C<__LINES__>
 items.  Items not present in the article will contain C<None>.
 
-C<art['__BODY__']> is a buffer object containing the article's entire body, and
-C<art['__LINES__']> is an int holding B<innd>'s reckoning of the number of lines
-in the article.  All the other elements will be buffers with the contents
-of the same-named article headers.
+C<art['__BODY__']> is a buffer/memoryview object containing the article's
+entire body, and C<art['__LINES__']> is a long integer holding B<innd>'s
+reckoning of the number of lines in the article.  All the other elements
+will be buffer/memoryview objects with the contents of the same-named
+article headers.
 
 The Newsgroups: header of the article is accessible inside the Python
 filter as C<art['Newsgroups']>.
@@ -111,13 +112,13 @@
     # Syntax for Python 2.x.
     Newsgroups = intern("Newsgroups")
     if art[Newsgroups] == buffer("misc.test"):
-        print("Test group")
+        syslog("n", "Test group")
 
     # Syntax for Python 3.x.
     import sys
     Newsgroups = sys.intern("Newsgroups")
     if art[Newsgroups] == memoryview(b"misc.test"):
-        print("Test group")
+        syslog("n", "Test group")
 
 If you want to accept an article, return C<None> or an empty string.  To
 reject, return a non-empty string.  The rejection strings will be shown to
@@ -127,10 +128,11 @@
 
 =item filter_messageid(I<self>, I<msgid>)
 
-I<msgid> is a buffer object containing the ID of an article being offered by
-CHECK, IHAVE or TAKETHIS.  Like with C<filter_art>, the message will be refused if
-you return a non-empty string.  If you use this feature, keep it light
-because it is called at a rather busy place in B<innd>'s main loop.
+I<msgid> is a string containing the ID of an article being offered
+by CHECK, IHAVE or TAKETHIS.  Like with C<filter_art>, the message will
+be refused if you return a non-empty string (properly encoded in UTF-8).
+If you use this feature, keep it light because it is called at a rather
+busy place in B<innd>'s main loop.
 
 =item filter_mode(I<self>, I<oldmode>, I<newmode>, I<reason>)
 
@@ -185,13 +187,13 @@
 whether the methods exist and are callable, but if you define one and get the
 parameter counts wrong, B<innd> WILL DIE.  You have been warned.  Be careful
 with your return values, too.  The C<filter_art> and C<filter_messageid>
-methods have to return strings, or C<None>.  If you return something like an
-int, B<innd> will I<not> be happy.
+methods have to return strings (encoded in UTF-8), or C<None>.  If you return
+something like an int, B<innd> will I<not> be happy.
 
 =head2 A Note regarding Buffer Objects
 
 This section is not applicable to S<Python 3.x> where buffer objects have
-been replaced with memory views.
+been replaced with memoryview objects.
 
 Buffer objects are cousins of strings, new in S<Python 1.5.2>.  Using buffer
 objects may take some getting used to, but we can create buffers much faster
@@ -304,23 +306,26 @@
     # We can look at the header or all of an article already on spool,
     # too.  Might be useful for long-memory despamming or
     # authentication things.  Each is returned (if present) as a
-    # string object; otherwise you'll end up with an empty string.
+    # string object (in Python 2.x) or a bytes object (in Python 3.x);
+    # otherwise you'll end up with an empty string.
     artbody = INN.article('<foo$bar.baz at bungmunch.edu>')
     artheader = INN.head('<foo$bar.baz at bungmunch.edu>')
 
     # As we can compute a hash digest for a string, we can obtain one
-    # for artbody.  It might be of help to detect spam.
+    # for artbody.  It might be of help to detect spam.  The digest is a
+    # string object (in Python 2.x) or a bytes object (in Python 3.x).
     digest = INN.hashstring(artbody)
 
     # Finally, do you want to see if a given newsgroup is moderated or
     # whatever?  INN.newsgroup returns the last field of a group's
-    # entry in active as a string.
+    # entry in active as a string object (in Python 2.x) or a bytes
+    # object (in Python 3.x).
     groupstatus = INN.newsgroup('alt.fan.karl-malden.nose')
-    if groupstatus == '':
+    if groupstatus == '':     # Compare to b'' for Python 3.x.
         moderated = 'no such newsgroup'
-    elif groupstatus == 'y':
+    elif groupstatus == 'y':  # Compare to b'y' for Python 3.x.
         moderated = "nope"
-    elif groupstatus == 'm':
+    elif groupstatus == 'm':  # Compare to b'm' for Python 3.x.
         moderated = "yep"
     else:
         moderated = "something else"
@@ -334,11 +339,12 @@
 <erik at eriq.org>; bug reports should however go to <inn-workers at lists.isc.org>,
 not Erik.
 
-The remainder of this section is an introduction to the new mechanism
-(which uses the I<python_auth>, I<python_access>, and I<python_dynamic>
-F<readers.conf> parameters) with porting/migration suggestions for
-people familiar with the old mechanism (identifiable by the now
-deprecated I<nnrpperlauth> parameter in F<inn.conf>).
+The remainder of this section is an introduction to the new
+mechanism introduced in S<INN 2.4.0> (which uses the I<python_auth>,
+I<python_access>, and I<python_dynamic> F<readers.conf> parameters)
+with porting/migration suggestions for people familiar with the
+old mechanism (identifiable by the now deprecated I<nnrppythonauth>
+parameter in F<inn.conf>).
 
 Other people should skip this section.
 
@@ -423,7 +429,7 @@
 inclusion of a I<python_auth> parameter in a F<readers.conf> auth
 group.  I<python_auth> works exactly like the I<auth> parameter in
 F<readers.conf>, except that it calls the script given as argument
-using the Python hook rather then treating it as an external
+using the Python hook rather than treating it as an external
 program.  Multiple, mixed use of I<python_auth> with other I<auth>
 statements including I<perl_auth> is permitted.  Each I<auth> statement
 will be tried in the order they appear in the auth group until either
@@ -521,9 +527,10 @@
 
 Called when a I<python_auth> statement is reached in the processing of
 F<readers.conf>.  Connection attributes are passed in the I<attributes>
-dictionary.  Returns a response code, an error string, and an optional
-string to be used in place of the client-supplied username (both for
-logging and for matching the connection with an access group).
+dictionary.  Returns a response code (as an integer), an error string
+(encoded in UTF-8), and an optional string (encoded in UTF-8) to be
+used in place of the client-supplied username (both for logging and
+for matching the connection with an access group).
 
 The NNTP response code should be 281 (authentication successful),
 481 (authentication unsuccessful), or 403 (server failure).  If the
@@ -550,8 +557,8 @@
 
 Called when a I<python_access> statement is reached in the processing of
 F<readers.conf>.  Connection attributes are passed in the I<attributes>
-dictionary.  Returns a dictionary of values representing statements to
-be included in an access group.
+dictionary.  Returns a dictionary of values (encoded in UTF-8) representing
+statements to be included in an access group.
 
 =item access_close(I<self>)
 
@@ -567,8 +574,10 @@
 
 Called when a client requests a newsgroup, an article or attempts to
 post.  Connection attributes are passed in the I<attributes> dictionary.
-Returns C<None> to grant access, or a non-empty string (which will be
-reported back to the client) otherwise.
+Returns C<None> to grant access, or a non-empty string encoded in UTF-8
+(which will be reported back to the client in response to GROUP or POST,
+and in any case logged in news logs files for all relevant NNTP commands)
+otherwise.
 
 =item dynamic_close(I<self>)
 
@@ -631,8 +640,9 @@
 
 =back
 
-All the above values are buffer objects (see the notes above on what
-buffer objects are).
+All the above values are buffer objects (see the notes above on
+what buffer objects are) for S<Python 2.x> or memoryview objects for
+S<Python 3.x>.
 
 =head2 How to Use these Methods with B<nnrpd>
 

Modified: doc/pod/install.pod
===================================================================
--- doc/pod/install.pod	2018-05-14 12:33:11 UTC (rev 10281)
+++ doc/pod/install.pod	2018-05-14 12:42:14 UTC (rev 10282)
@@ -146,7 +146,7 @@
 versions you'll need:
 
     --with-perl         Perl 5.004_03 or higher, 5.8.0+ recommended
-    --with-python       Python 2.2.0 or higher, 2.5.0+ recommended (3.x versions currently not supported)
+    --with-python       Python 2.3.0 or higher, 2.5.0+ recommended (in the 2.x series); Python 3.3.0 or higher (in the 3.x series)
     --with-bdb          Berkeley DB 4.4 or higher, 4.7+ recommended
     --with-zlib         zlib 1.x or higher
     --with-openssl      OpenSSL 0.9.6 or higher
@@ -350,10 +350,13 @@
 
 Enables support for Python, allowing you to install filter and
 authentication scripts written in Python.  You will need S<Python
-2.2.0> or later installed on your system to enable this option.
-See F<doc/hook-python> for all the details.  Please note that S<Python
-3.x> is currently not supported.
+2.3.0> or later (in the 2.x series), or S<Python 3.3.0> or later
+(in the 3.x series) installed on your system to enable this option.
+See F<doc/hook-python> for all the details.
 
+If the C<$PYTHON> environment variable is set, it will be used as the path
+to Python.
+
 =item B<--with-innd-port>=PORT
 
 By default, innbind(8) refuses to bind to any port under 1024 other than

Modified: doc/pod/news.pod
===================================================================
--- doc/pod/news.pod	2018-05-14 12:33:11 UTC (rev 10281)
+++ doc/pod/news.pod	2018-05-14 12:42:14 UTC (rev 10282)
@@ -1,3 +1,30 @@
+=head1 Changes in 2.6.3
+
+=over 2
+
+=item *
+
+Support for S<Python 3> has been added to INN.  Embedded Python filtering
+and authentication hooks for B<innd> and B<nnrpd> can now use S<version
+3.3.0> or later of the Python interpreter.  In the 2.x series, S<version
+2.3.0> or later is still supported.
+
+When configuring INN with the B<--with-python> flag, the C<PYTHON>
+environment variable, when set, is used to select the interpreter
+to embed.  Otherwise, it is searched in standard paths.
+
+In case you change the Python interpreter to embed, make sure that
+the Python scripts you use are written in the expected syntax for that
+version of the Python interpreter.  Notably, buffer objects have been
+replaced with memoryview objects in S<Python 3>, and UTF-8 encoding
+now really matters for string literals (S<Python 3> uses bytes and
+Unicode objects).
+
+INN documentation and samples of Python hooks have been updated to
+provide more examples.
+
+=back
+
 =head1 Changes in 2.6.2
 
 =over 2

Modified: innd/innd.h
===================================================================
--- innd/innd.h	2018-05-14 12:33:11 UTC (rev 10281)
+++ innd/innd.h	2018-05-14 12:42:14 UTC (rev 10282)
@@ -862,6 +862,11 @@
 			       char *reason);
 extern void		PYsetup(void);
 extern void		PYclose(void);
+# if PY_MAJOR_VERSION >= 3
+extern PyMODINIT_FUNC   PyInit_INN(void);
+# else
+extern void             PyInit_INN(void);
+# endif
 #endif /* DO_PYTHON */
 
 END_DECLS

Modified: innd/python.c
===================================================================
--- innd/python.c	2018-05-14 12:33:11 UTC (rev 10281)
+++ innd/python.c	2018-05-14 12:42:14 UTC (rev 10282)
@@ -41,6 +41,18 @@
   typedef int Py_ssize_t;
 #endif
 
+#if PY_MAJOR_VERSION >= 3
+# define PyInt_FromLong PyLong_FromLong
+# define PyString_AS_STRING PyUnicode_AsUTF8
+# define PyString_FromStringAndSize PyBytes_FromStringAndSize
+# define PyString_InternFromString PyUnicode_InternFromString
+# define PYBUFF_FROMMEMORY(str, len) \
+      PyMemoryView_FromMemory((str), (len), PyBUF_WRITE)
+#else
+# define PYBUFF_FROMMEMORY(str, len) \
+      PyBuffer_FromMemory((str), (len))
+#endif
+
 #include "clibrary.h"
 
 #include "inn/innconf.h"
@@ -56,7 +68,7 @@
 PyObject	*PYheaders = NULL;
 PyObject	**PYheaditem;
 PyObject	**PYheadkey;
-PyObject	*PYpathkey, *PYlineskey, *PYbodykey;
+PyObject	*PYlineskey, *PYbodykey;
 
 /*  External functions. */
 PyObject	*msgid_method = NULL;
@@ -126,7 +138,7 @@
     hdrnum = 0;
     for (i = 0 ; i < MAX_ARTHEADER ; i++) {
 	if (HDR_FOUND(i)) {
-	    PYheaditem[hdrnum] = PyBuffer_FromMemory(HDR(i), HDR_LEN(i));
+            PYheaditem[hdrnum] = PYBUFF_FROMMEMORY(HDR(i), HDR_LEN(i));
 	} else
 	    PYheaditem[hdrnum] = Py_None;
 	PyDict_SetItem(PYheaders, PYheadkey[hdrnum], PYheaditem[hdrnum]);
@@ -135,7 +147,7 @@
 
     /* ...then the body... */
     if (artLen && artBody != NULL)
-        PYheaditem[hdrnum] = PyBuffer_FromMemory(artBody, --artLen);
+        PYheaditem[hdrnum] = PYBUFF_FROMMEMORY(artBody, --artLen);
     else
         PYheaditem[hdrnum] = Py_None;
     PyDict_SetItem(PYheaders, PYbodykey, PYheaditem[hdrnum++]);
@@ -564,8 +576,38 @@
     METHOD(NULL,              NULL,               0,            "")
 };
 
+#if PY_MAJOR_VERSION >= 3
+static struct PyModuleDef INNPyModule = {
+    PyModuleDef_HEAD_INIT,                /* m_base */
+    (char *) "INN",                       /* m_name */
+    (char *) "innd Python filter hook",   /* m_doc */
+    -1,                                   /* m_size */
+    INNPyMethods,                         /* m_methods */
+    NULL,                                 /* m_slots */
+    NULL,                                 /* m_traverse */
+    NULL,                                 /* m_clear */
+    NULL,                                 /* m_free */
+};
 
+PyMODINIT_FUNC
+PyInit_INN(void) {
+    PyObject *module = PyModule_Create(&INNPyModule);
 
+    if (module == NULL)
+        syslog(L_ERROR, "failed to create innd python module");
+
+    return module;
+}
+#else
+void
+PyInit_INN(void) {
+    if (Py_InitModule3((char *) "INN", INNPyMethods,
+                       (char *) "innd Python filter hook") == NULL)
+        syslog(L_ERROR, "failed to initialize innd python module");
+}
+#endif
+
+
 /*
 **  This runs when innd shuts down.
 */
@@ -699,11 +741,14 @@
     const ARTHEADER *hp;
     size_t hdrcount;
 
-    /* Add path for nnrpd module.  The environment variable PYTHONPATH
+    /* Add path for innd module.  The environment variable PYTHONPATH
      * does it; one can also append innconf->pathfilter to sys.path once
      * Python has been initialized. */
     setenv("PYTHONPATH", innconf->pathfilter, 1);
 
+    /* Build a module interface to certain INN functions. */
+    PyImport_AppendInittab("INN", &PyInit_INN);
+
     /* Load up the interpreter ;-O */
     Py_Initialize();
 
@@ -718,12 +763,10 @@
 	return;
     }
 
-    /* Build a module interface to certain INN functions. */
-    Py_InitModule((char *) "INN", INNPyMethods);
-
     PYFilterModule = PyImport_ImportModule((char *) INN_PATH_PYTHON_STARTUP_M);
     if (PYFilterModule == NULL)
-	syslog(L_ERROR, "failed to import external python module");
+	syslog(L_ERROR, "failed to import external %s python module",
+           INN_PATH_PYTHON_STARTUP_M);
 
     if (PYFilterObject == NULL) {
 	syslog(L_ERROR, "python filter object is not defined");
@@ -742,10 +785,9 @@
     PYheaditem = xmalloc((hdrcount + 2) * sizeof(PyObject *));
     PYheadkey = xmalloc(hdrcount * sizeof(PyObject *));
 
-    /* Preallocate keys for the article dictionary */
+    /* Preallocate keys for the article dictionary. */
     for (hp = ARTheaders; hp < ARRAY_END(ARTheaders); hp++)
 	PYheadkey[hp - ARTheaders] = PyString_InternFromString(hp->Name);
-    PYpathkey = PyString_InternFromString("Path");
     PYlineskey = PyString_InternFromString("__LINES__");
     PYbodykey = PyString_InternFromString("__BODY__");
 

Modified: m4/python.m4
===================================================================
--- m4/python.m4	2018-05-14 12:33:11 UTC (rev 10281)
+++ m4/python.m4	2018-05-14 12:42:14 UTC (rev 10282)
@@ -1,6 +1,25 @@
-dnl python.m4 -- Probe for the details needed to embed Python.
+dnl Probe for Python properties and, optionally, flags for embedding Python.
 dnl $Id$
 dnl
+dnl Provides the following macros:
+dnl
+dnl INN_PROG_PYTHON
+dnl     Checks for a specific Python version and sets the PYTHON environment
+dnl     variable to the full path, or aborts the configure run if the version
+dnl     of Python is not new enough or couldn't be found.
+dnl     The first argument is a Python version related to the 2.x series (if
+dnl     empty, it means that Python 2 is not supported).  The second argument
+dnl     is a Python version related to at least the 3.x series (if empty,
+dnl     it means that Python 3 or later is not supported).
+dnl
+dnl INN_PYTHON_CHECK_MODULE
+dnl     Checks for the existence of a Python module and runs provided code
+dnl     based on whether or not it was found.
+dnl
+dnl INN_LIB_PYTHON
+dnl     Determines the flags required for embedding Python and sets
+dnl     PYTHON_CPPFLAGS and PYTHON_LIBS.
+dnl
 dnl Defines INN_ARG_PYTHON, which sets up the --with-python command line
 dnl argument and also sets various flags needed for embedded Python if it is
 dnl requested.
@@ -7,37 +26,63 @@
 dnl
 dnl We use the distutils.sysconfig module shipped with Python 2.2.0 and later
 dnl to find the compiler and linker flags to use to embed Python.
+dnl We also select libpython in the main library location (a shared library
+dnl is present there in Python 2.3.0 and later).
 
-AC_DEFUN([INN_ARG_PYTHON],
+dnl Check for the path to Python and ensure it meets our minimum version
+dnl requirement (given as the argument).  Honor the $PYTHON environment
+dnl variable, if set.
+AC_DEFUN([INN_PROG_PYTHON],
 [AC_ARG_VAR([PYTHON], [Location of Python interpreter])
- AC_ARG_WITH([python],
-    [AS_HELP_STRING([--with-python], [Embedded Python module support [no]])],
-    [AS_CASE([$withval],
-     [yes], [DO_PYTHON=DO
-             AC_DEFINE([DO_PYTHON], [1],
-                [Define to compile in Python module support.])],
-     [no], [DO_PYTHON=DONT],
-     [AC_MSG_ERROR([invalid argument to --with-python])])],
-    [DO_PYTHON=DONT])
- AS_IF([test x"$DO_PYTHON" = xDO],
-    [INN_PATH_PROG_ENSURE([PYTHON], [python])
-     AC_MSG_CHECKING([for Python linkage])
-     py_include=`$PYTHON -c 'import distutils.sysconfig; \
-         print(distutils.sysconfig.get_python_inc())'`
-     PYTHON_CPPFLAGS="-I$py_include"
-     py_ver=`$PYTHON -c 'import sys; print(sys.version[[:3]])'`
-     py_libdir=`$PYTHON -c 'import distutils.sysconfig; \
-         print(distutils.sysconfig.get_python_lib(0, 1))'`
-     py_linkage=`$PYTHON -c 'import distutils.sysconfig; \
-         print(" ".join(distutils.sysconfig.get_config_vars("LIBS", \
-             "LIBC", "LIBM", "LOCALMODLIBS", "BASEMODLIBS", \
-             "LINKFORSHARED", "LDFLAGS")))'`
-     py_configdir=`$PYTHON -c 'import distutils.sysconfig; \
-         print(distutils.sysconfig.get_config_var("LIBPL"))'`
-     PYTHON_LIBS="-L$py_configdir -lpython$py_ver $py_linkage"
-     PYTHON_LIBS=`echo $PYTHON_LIBS | sed -e 's/[ \\t]*/ /g'`
-     AC_MSG_RESULT([$py_libdir])],
-    [PYTHON_CPPFLAGS=
-     PYTHON_LIBS=])
- AC_SUBST([PYTHON_CPPFLAGS])
- AC_SUBST([PYTHON_LIBS])])
+ AS_IF([test x"$1" != x], [py_expected_ver="$1 (in the 2.x series)"],
+     [py_expected_ver=""])
+ AS_IF([test x"$2" != x],
+     [AS_IF([test x"$1" != x], [py_expected_ver="$py_expected_ver or "])
+      py_expected_ver="${py_expected_ver}$2"])
+ AS_IF([test x"$PYTHON" != x],
+    [AS_IF([! test -x "$PYTHON"],
+        [AC_MSG_ERROR([Python binary $PYTHON not found])])
+     AS_IF([! "$PYTHON" -c 'import sys; assert((False if "$2" == "" else (sys.version_info.major > 2 and sys.version_info >= tuple(int(i) for i in "$2".split(".")))) if "$1" == "" else ((sys.version_info.major == 2 and sys.version_info >= tuple(int(i) for i in "$1".split("."))) if "$2" == "" else ((sys.version_info.major == 2 and sys.version_info >= tuple(int(i) for i in "$1".split("."))) or sys.version_info >= tuple(int(i) for i in "$2".split(".")))))' >/dev/null 2>&1],
+        [AC_MSG_ERROR([Python $py_expected_ver or greater is required])])],
+    [AC_CACHE_CHECK([for Python version $py_expected_ver or later], [ac_cv_path_PYTHON],
+        [AC_PATH_PROGS_FEATURE_CHECK([PYTHON], [python],
+            [AS_IF(["$ac_path_PYTHON" -c 'import sys; assert((False if "$2" == "" else (sys.version_info.major > 2 and sys.version_info >= tuple(int(i) for i in "$2".split(".")))) if "$1" == "" else ((sys.version_info.major == 2 and sys.version_info >= tuple(int(i) for i in "$1".split("."))) if "$2" == "" else ((sys.version_info.major == 2 and sys.version_info >= tuple(int(i) for i in "$1".split("."))) or sys.version_info >= tuple(int(i) for i in "$2".split(".")))))' >/dev/null 2>&1],
+                [ac_cv_path_PYTHON="$ac_path_PYTHON"
+                 ac_path_PYTHON_found=:])])])
+     AS_IF([test x"$ac_cv_path_PYTHON" = x],
+         [AC_MSG_ERROR([Python $py_expected_ver or greater is required])])
+     PYTHON="$ac_cv_path_PYTHON"
+     AC_SUBST([PYTHON])])])
+
+dnl Check whether a given Python module can be loaded.  Runs the second argument
+dnl if it can, and the third argument if it cannot.
+AC_DEFUN([INN_PYTHON_CHECK_MODULE],
+[AS_LITERAL_IF([$1], [], [m4_fatal([$0: requires literal arguments])])dnl
+ AS_VAR_PUSHDEF([ac_Module], [inn_cv_python_module_$1])dnl
+ AC_CACHE_CHECK([for Python module $1], [ac_Module],
+    [AS_IF(["$PYTHON" -c 'import $1' >/dev/null 2>&1],
+        [AS_VAR_SET([ac_Module], [yes])],
+        [AS_VAR_SET([ac_Module], [no])])])
+ AS_VAR_IF([ac_Module], [yes], [$2], [$3])
+ AS_VAR_POPDEF([ac_Module])])
+
+dnl Determine the flags used for embedding Python.
+AC_DEFUN([INN_LIB_PYTHON],
+[AC_SUBST([PYTHON_CPPFLAGS])
+ AC_SUBST([PYTHON_LIBS])
+ AC_MSG_CHECKING([for flags to link with Python])
+ py_include=`$PYTHON -c 'import distutils.sysconfig; \
+     print(distutils.sysconfig.get_python_inc())'`
+ PYTHON_CPPFLAGS="-I$py_include"
+ py_libdir=`$PYTHON -c 'import distutils.sysconfig; \
+     print(" -L".join(distutils.sysconfig.get_config_vars("LIBDIR")))'`
+ py_ldlibrary=`$PYTHON -c 'import distutils.sysconfig; \
+     print(distutils.sysconfig.get_config_vars("LDLIBRARY")@<:@0@:>@)'`
+ py_linkage=`$PYTHON -c 'import distutils.sysconfig; \
+     print(" ".join(distutils.sysconfig.get_config_vars("LIBS", \
+         "LIBC", "LIBM", "LOCALMODLIBS", "BASEMODLIBS", \
+         "LINKFORSHARED", "LDFLAGS")))'`
+ py_libpython=`echo $py_ldlibrary | sed "s/^lib//" | sed "s/\.@<:@a-z@:>@*$//"`
+ PYTHON_LIBS="-L$py_libdir -l$py_libpython $py_linkage"
+ PYTHON_LIBS=`echo $PYTHON_LIBS | sed -e 's/[ \\t]*/ /g'`
+ AC_MSG_RESULT([$PYTHON_LIBS])])

Modified: nnrpd/nnrpd.h
===================================================================
--- nnrpd/nnrpd.h	2018-05-14 12:33:11 UTC (rev 10281)
+++ nnrpd/nnrpd.h	2018-05-14 12:42:14 UTC (rev 10282)
@@ -308,6 +308,11 @@
 void PY_close_python(void);
 int PY_dynamic(char *Username, char *NewsGroup, int PostFlag, char **reply_message);
 void PY_dynamic_init (char* file);
+# if PY_MAJOR_VERSION >= 3
+extern PyMODINIT_FUNC   PyInit_nnrpd(void);
+# else
+extern void             PyInit_nnrpd(void);
+# endif
 #endif	/* DO_PYTHON */
 
 void line_free(struct line *);

Modified: nnrpd/python.c
===================================================================
--- nnrpd/python.c	2018-05-14 12:33:11 UTC (rev 10281)
+++ nnrpd/python.c	2018-05-14 12:42:14 UTC (rev 10282)
@@ -42,6 +42,20 @@
   typedef int Py_ssize_t;
 #endif
 
+#if PY_MAJOR_VERSION >= 3
+# define PyInt_AS_LONG PyLong_AS_LONG
+# define PyInt_Check PyLong_Check
+# define PyInt_FromLong PyLong_FromLong
+# define PyString_AS_STRING PyUnicode_AsUTF8
+# define PyString_AsString PyUnicode_AsUTF8
+# define PyString_Check PyUnicode_Check
+# define PYBUFF_FROMMEMORY(str, len) \
+      PyMemoryView_FromMemory((str), (len), PyBUF_WRITE)
+#else
+# define PYBUFF_FROMMEMORY(str, len) \
+      PyBuffer_FromMemory((str), (len))
+#endif
+
 #include "clibrary.h"
 
 #include "inn/innconf.h"
@@ -137,11 +151,11 @@
     authnum = 0;
 
     /* Client hostname. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.host, strlen(Client.host));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.host, strlen(Client.host));
     PyDict_SetItemString(PYauthinfo, PYTHONhostname, PYauthitem[authnum++]);
 
     /* Client IP number. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.ip, strlen(Client.ip));
+    PYauthitem[authnum] =PYBUFF_FROMMEMORY(Client.ip, strlen(Client.ip));
     PyDict_SetItemString(PYauthinfo, PYTHONipaddress, PYauthitem[authnum++]);
 
     /* Client port number. */
@@ -149,11 +163,13 @@
     PyDict_SetItemString(PYauthinfo, PYTHONport, PYauthitem[authnum++]);
 
     /* Server interface the connection comes to. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.serverhost, strlen(Client.serverhost));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.serverhost,
+                                            strlen(Client.serverhost));
     PyDict_SetItemString(PYauthinfo, PYTHONinterface, PYauthitem[authnum++]);
 
     /* Server IP number. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.serverip, strlen(Client.serverip));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.serverip,
+                                            strlen(Client.serverip));
     PyDict_SetItemString(PYauthinfo, PYTHONintipaddr, PYauthitem[authnum++]);
 
     /* Server port number. */
@@ -164,7 +180,7 @@
     if (Username == NULL) {
         PYauthitem[authnum] = Py_None;
     } else {
-        PYauthitem[authnum] = PyBuffer_FromMemory(Username, strlen(Username));
+        PYauthitem[authnum] = PYBUFF_FROMMEMORY(Username, strlen(Username));
     }
     PyDict_SetItemString(PYauthinfo, PYTHONuser, PYauthitem[authnum++]);
 
@@ -172,7 +188,7 @@
     if (Password == NULL) {
         PYauthitem[authnum] = Py_None;
     } else {
-        PYauthitem[authnum] = PyBuffer_FromMemory(Password, strlen(Password));
+        PYauthitem[authnum] = PYBUFF_FROMMEMORY(Password, strlen(Password));
     }
     PyDict_SetItemString(PYauthinfo, PYTHONpass, PYauthitem[authnum++]);
 
@@ -281,11 +297,11 @@
     authnum = 0;
 
     /* Client hostname. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.host, strlen(Client.host));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.host, strlen(Client.host));
     PyDict_SetItemString(PYauthinfo, PYTHONhostname, PYauthitem[authnum++]);
 
     /* Client IP number. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.ip, strlen(Client.ip));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.ip, strlen(Client.ip));
     PyDict_SetItemString(PYauthinfo, PYTHONipaddress, PYauthitem[authnum++]);
 
     /* Client port number. */
@@ -293,11 +309,13 @@
     PyDict_SetItemString(PYauthinfo, PYTHONport, PYauthitem[authnum++]);
 
     /* Server interface the connection comes to. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.serverhost, strlen(Client.serverhost));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.serverhost,
+                                            strlen(Client.serverhost));
     PyDict_SetItemString(PYauthinfo, PYTHONinterface, PYauthitem[authnum++]);
 
     /* Server IP number. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.serverip, strlen(Client.serverip));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.serverip,
+                                            strlen(Client.serverip));
     PyDict_SetItemString(PYauthinfo, PYTHONintipaddr, PYauthitem[authnum++]);
 
     /* Server port number. */
@@ -305,7 +323,7 @@
     PyDict_SetItemString(PYauthinfo, PYTHONintport, PYauthitem[authnum++]);
 
     /* Username. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Username, strlen(Username));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Username, strlen(Username));
     PyDict_SetItemString(PYauthinfo, PYTHONuser, PYauthitem[authnum++]);
  
     /* Password is not known. */
@@ -410,11 +428,11 @@
     authnum = 0;
 
     /* Client hostname. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.host, strlen(Client.host));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.host, strlen(Client.host));
     PyDict_SetItemString(PYauthinfo, PYTHONhostname, PYauthitem[authnum++]);
 
     /* Client IP number. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.ip, strlen(Client.ip));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.ip, strlen(Client.ip));
     PyDict_SetItemString(PYauthinfo, PYTHONipaddress, PYauthitem[authnum++]);
 
     /* Client port number. */
@@ -422,11 +440,13 @@
     PyDict_SetItemString(PYauthinfo, PYTHONport, PYauthitem[authnum++]);
 
     /* Server interface the connection comes to. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.serverhost, strlen(Client.serverhost));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.serverhost,
+                                            strlen(Client.serverhost));
     PyDict_SetItemString(PYauthinfo, PYTHONinterface, PYauthitem[authnum++]);
 
     /* Server IP number. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Client.serverip, strlen(Client.serverip));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Client.serverip,
+                                            strlen(Client.serverip));
     PyDict_SetItemString(PYauthinfo, PYTHONintipaddr, PYauthitem[authnum++]);
 
     /* Server port number. */
@@ -434,7 +454,7 @@
     PyDict_SetItemString(PYauthinfo, PYTHONintport, PYauthitem[authnum++]);
     
     /* Username. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(Username, strlen(Username));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(Username, strlen(Username));
     PyDict_SetItemString(PYauthinfo, PYTHONuser, PYauthitem[authnum++]);
     
     /* Password is not known. */
@@ -443,11 +463,11 @@
 
     /* Assign authentication type. */
     PYauthitem[authnum] =
-        PyBuffer_FromMemory((char *)(PostFlag ? "post" : "read"), 4);
+        PYBUFF_FROMMEMORY((char *)(PostFlag ? "post" : "read"), 4);
     PyDict_SetItemString(PYauthinfo, PYTHONtype, PYauthitem[authnum++]);
  
     /* Newsgroup user tries to access. */
-    PYauthitem[authnum] = PyBuffer_FromMemory(NewsGroup, strlen(NewsGroup));
+    PYauthitem[authnum] = PYBUFF_FROMMEMORY(NewsGroup, strlen(NewsGroup));
     PyDict_SetItemString(PYauthinfo, PYTHONnewsgroup,  PYauthitem[authnum++]);
     
     /*
@@ -548,10 +568,10 @@
 PY_syslog(PyObject *self UNUSED, PyObject *args)
 {
     char        *loglevel;
-    int         levellen;
+    Py_ssize_t   levellen;
     char        *logmsg;
-    int         msglen;
-    int         priority;
+    Py_ssize_t   msglen;
+    int          priority;
 
     /* Get loglevel and message. */
     if (!PyArg_ParseTuple(args, (char *) "s#s#", &loglevel, &levellen,
@@ -595,8 +615,38 @@
     METHOD(NULL,            NULL,             0,            "")
 };
 
+#if PY_MAJOR_VERSION >= 3
+static struct PyModuleDef nnrpdPyModule = {
+    PyModuleDef_HEAD_INIT,                /* m_base */
+    (char *) "nnrpd",                     /* m_name */
+    (char *) "nnrpd Python filter hook",  /* m_doc */
+    -1,                                   /* m_size */
+    nnrpdPyMethods,                       /* m_methods */
+    NULL,                                 /* m_slots */
+    NULL,                                 /* m_traverse */
+    NULL,                                 /* m_clear */
+    NULL,                                 /* m_free */
+};
 
+PyMODINIT_FUNC
+PyInit_nnrpd(void) {
+    PyObject *module = PyModule_Create(&nnrpdPyModule);
 
+    if (module == NULL)
+        syslog(L_ERROR, "failed to create nnrpd python module");
+
+    return module;
+}
+#else
+void
+PyInit_nnrpd(void) {
+    if (Py_InitModule3((char *) "nnrpd", nnrpdPyMethods,
+                       (char *) "nnrpd Python filter hook") == NULL)
+        syslog(L_ERROR, "failed to initialize nnrpd python module");
+}
+#endif
+
+
 /*
 **  Called by the external module so it can register itself with nnrpd.
 */
@@ -635,6 +685,9 @@
         */
         setenv("PYTHONPATH", innconf->pathfilter, 1);
 
+        /* Build a module interface to certain nnrpd functions. */
+        PyImport_AppendInittab("nnrpd", &PyInit_nnrpd);
+
         /* Load up the interpreter ;-O */
         Py_Initialize();
     
@@ -648,9 +701,6 @@
             return;
         }
 
-        /* Build a module interface to certain nnrpd functions. */
-        Py_InitModule((char *) "nnrpd", nnrpdPyMethods);
-
         /*
         ** Grab space for authinfo dictionary so we aren't forever
         ** recreating them.

Modified: samples/filter_innd.py
===================================================================
--- samples/filter_innd.py	2018-05-14 12:33:11 UTC (rev 10281)
+++ samples/filter_innd.py	2018-05-14 12:42:14 UTC (rev 10282)
@@ -18,11 +18,11 @@
 
 import re
 from string import *
+import sys
 
 
 ##  The built-in intern() method has been in the sys module
 ##  since Python 3.0.
-import sys
 if sys.version_info[0] >= 3:
     def intern(headerName):
         return sys.intern(headerName)
@@ -157,6 +157,8 @@
         """
         return ""               # Deactivate the samples.
 
+        syslog('notice', "just seen %s" % msgid)
+
         if self.re_none44.search(msgid):
             return "But I don't like spam!"
         if msgid[0:8] == '<cancel.':
@@ -173,10 +175,10 @@
         innd/art.c.  At this writing, they are:
 
             Also-Control, Approved, Archive, Archived-At, Bytes, Cancel-Key, Cancel-Lock,
-            Content-Base, Content-Disposition, Content-Transfer-Encoding,
+            Comments, Content-Base, Content-Disposition, Content-Transfer-Encoding,
             Content-Type, Control, Date, Date-Received, Distribution, Expires,
             Face, Followup-To, From, In-Reply-To, Injection-Date, Injection-Info,
-            Keywords, Lines, List-ID, Message-ID, MIME-Version, Newsgroups,
+            Jabber-ID, Keywords, Lines, List-ID, Message-ID, MIME-Version, Newsgroups,
             NNTP-Posting-Date, NNTP-Posting-Host, NNTP-Posting-Path,
             Organization, Original-Sender, Originator,
             Path, Posted, Posting-Version, Received, References, Relay-Version,
@@ -202,7 +204,24 @@
         """
         return ""               # Deactivate the samples.
 
-        # Catch bad Message-IDs from articles fed with TAKETHIS but no CHECK.
+        # Example of decoding the Newsgroups: header field with Python 3.x
+        # using bytes object.
+        #  header = art[Newsgroups].tobytes().decode("utf-8")
+        #  syslog('notice', "Newsgroups: %s" % header)
+        #
+        # Another example with the Distribution: header field, that may not
+        # be present in the headers, and also not in UTF-8.
+        #  if art[Distribution]:
+        #      header = art[Distribution].tobytes()
+        #      syslog('notice', "Distribution: %s" % header)
+        #
+        # Other examples:
+        #  syslog('notice', "Article body: %s" % art[__BODY__].tobytes())
+        #  syslog('notice', "Number of lines: %lu" % art[__LINES__])
+
+        # Catch bad Message-IDs from articles (in case Message-IDs provided
+        # as arguments to the IHAVE or TAKETHIS commands are not the real
+        # ones present in article headers).
         idcheck = self.filter_messageid(art[Message_ID])
         if idcheck:
             return idcheck
@@ -217,9 +236,11 @@
                     # Python 3.x uses memoryview(b'mxyzptlk') because buffers
                     # do not exist any longer.  Note that the argument is
                     # a bytes object.
-                    # if art[Distribution] == memoryview(b'mxyzptlk'):
-                    if art[Distribution] == buffer('mxyzptlk'):
-                        return "Evil control message from the 10th dimension"
+                    #  if art[Distribution] == memoryview(b'mxyzptlk'):
+                    #      return "Evil control message from the 10th dimension"
+                    # whereas in Python 2.x:
+                    #  if art[Distribution] == buffer('mxyzptlk'):
+                    #      return "Evil control message from the 10th dimension"
                 if self.re_obsctl.match(art[Control]):
                     return "Obsolete control message"
 
@@ -268,15 +289,7 @@
 ##  INN's.  Oh yeah -- you may notice that stdout and stderr have been
 ##  redirected to /dev/null -- if you want to print stuff, open your
 ##  own files.
-
-try:
-    import sys
-
-except Exception, errmsg:    # Syntax for Python 2.x.
-#except Exception as errmsg: # Syntax for Python 3.x.
-    syslog('Error', "import boo-boo: " + errmsg[0])
-
-
+##
 ##  If you want to do something special when the server first starts
 ##  up, this is how to find out when it's time.
 
@@ -296,6 +309,6 @@
 try:
     set_filter_hook(spamfilter)
     syslog('n', "spamfilter successfully hooked into INN")
-except Exception, errmsg:    # Syntax for Python 2.x.
-#except Exception as errmsg: # Syntax for Python 3.x.
-    syslog('e', "Cannot obtain INN hook for spamfilter: %s" % errmsg[0])
+except Exception: # Syntax valid in both Python 2.x and 3.x.
+    e = sys.exc_info()[1]
+    syslog('e', "Cannot obtain INN hook for spamfilter: %s" % e.args[0])

Modified: samples/nnrpd_access.py
===================================================================
--- samples/nnrpd_access.py	2018-05-14 12:33:11 UTC (rev 10281)
+++ samples/nnrpd_access.py	2018-05-14 12:42:14 UTC (rev 10282)
@@ -4,7 +4,7 @@
 ##
 ##  See the INN Python Filtering and Authentication Hooks documentation
 ##  for more information.
-##  The perl_access: parameter in readers.conf is used to load this script.
+##  The python_access: parameter in readers.conf is used to load this script.
 ##
 ##  An instance of ACCESS class is passed to nnrpd via the set_auth_hook()
 ##  function imported from nnrpd.  The following methods of that class
@@ -25,8 +25,8 @@
 ##                                your state variables or close a
 ##                                database connection.  May be omitted.
 ##
-##  If there is a problem with return codes from any of these methods, then nnrpd
-##  will die and syslog the exact reason.
+##  If there is a problem with return codes from any of these methods,
+##  then nnrpd will die and syslog the exact reason.
 ##
 ##  There are also a few Python functions defined in nnrpd:
 ##
@@ -53,21 +53,27 @@
     def access(self, attributes):
         """Called when python_access: is encountered in readers.conf."""
 
-        # Just for debugging purposes.
-        syslog('notice', 'n_a access() invoked: hostname %s, ipaddress %s, interface %s, user %s' % ( \
-                attributes['hostname'], \
-                attributes['ipaddress'], \
-                attributes['interface'], \
-                attributes['user']))
+        # Just for debugging purposes (in Python 3.x syntax).
+        # syslog('notice', 'n_a access() invoked: hostname %s, ipaddress %s, port %lu, interface %s, intipaddr %s, intport %lu, user %s' % ( \
+        #        attributes['hostname'].tobytes(), \
+        #        attributes['ipaddress'].tobytes(), \
+        #        attributes['port'], \
+        #        attributes['interface'].tobytes(), \
+        #        attributes['intipaddr'].tobytes(), \
+        #        attributes['intport'], \
+        #        (attributes['user'].tobytes() if attributes['user'] else "-")))
 
         # Allow newsreading from specific host only.
-        if '127.0.0.1' == str(attributes['ipaddress']):
-            syslog('notice', 'authentication access by IP address succeeded')
-            return {'read':'*', 'post':'*'}
-        else:
-            syslog('notice', 'authentication access by IP address failed')
-            return {'read':'!*', 'post':'!*'}
+        # Python 2.x syntax:
+        #  if '127.0.0.1' == str(attributes['ipaddress']):
+        # Python 3.x syntax:
+        #  if b'127.0.0.1' == attributes['ipaddress'].tobytes():
+        #    syslog('notice', 'authentication access by IP address succeeded')
+        #    return {'read':'*', 'post':'*'}
 
+        syslog('notice', 'authentication access by IP address failed')
+        return {'read':'!*', 'post':'!*'}
+
     def access_close(self):
         """Called on nnrpd termination."""
         pass
@@ -85,9 +91,10 @@
 
 ##  ...and try to hook up on nnrpd.  This would make auth object methods visible
 ##  to nnrpd.
+import sys
 try:
     set_auth_hook(myaccess)
     syslog('notice', "access module successfully hooked into nnrpd")
-except Exception, errmsg:    # Syntax for Python 2.x.
-#except Exception as errmsg: # Syntax for Python 3.x.
-    syslog('error', "Cannot obtain nnrpd hook for access method: %s" % errmsg[0])
+except Exception: # Syntax valid in both Python 2.x and 3.x.
+    e = sys.exc_info()[1]
+    syslog('error', "Cannot obtain nnrpd hook for access method: %s" % e.args[0])

Modified: samples/nnrpd_access_wrapper.py
===================================================================
--- samples/nnrpd_access_wrapper.py	2018-05-14 12:33:11 UTC (rev 10281)
+++ samples/nnrpd_access_wrapper.py	2018-05-14 12:42:14 UTC (rev 10282)
@@ -35,14 +35,16 @@
         # Python 3.x uses memoryview(b'connect') because buffers
         # do not exist any longer.  Note that the argument is
         # a bytes object.
-        # attributes['type'] = memoryview(b'connect')
-        attributes['type'] = buffer('connect')
-        perm = (self.old).authenticate(attributes)
+        #  attributes['type'] = memoryview(b'connect')
+        #  perm = (self.old).authenticate(attributes)
+        # whereas in Python 2.x:
+        #  attributes['type'] = buffer('connect')
+        #  perm = (self.old).authenticate(attributes)
         result = dict({'users': '*'})
-        if perm[1] == 1:
-            result['read'] = perm[3]
-        if perm[2] == 1:
-            result['post'] = perm[3]
+        #if perm[1] == 1:
+        #    result['read'] = perm[3]
+        #if perm[2] == 1:
+        #    result['post'] = perm[3]
         return result
 
     def access_close(self):
@@ -59,11 +61,12 @@
 ##  Create a class instance.
 myaccess = MYACCESS()
 
-##  ...and try to hook up on nnrpd.  This would make access object methods visible
-##  to nnrpd.
+##  ...and try to hook up on nnrpd.  This would make access object methods
+##  visible to nnrpd.
+import sys
 try:
     set_auth_hook(myaccess)
     syslog('notice', "access module successfully hooked into nnrpd")
-except Exception, errmsg:    # Syntax for Python 2.x.
-#except Exception as errmsg: # Syntax for Python 3.x.
-    syslog('error', "Cannot obtain nnrpd hook for access method: %s" % errmsg[0])
+except Exception: # Syntax valid in both Python 2.x and 3.x.
+    e = sys.exc_info()[1]
+    syslog('error', "Cannot obtain nnrpd hook for access method: %s" % e.args[0])

Modified: samples/nnrpd_auth.py
===================================================================
--- samples/nnrpd_auth.py	2018-05-14 12:33:11 UTC (rev 10281)
+++ samples/nnrpd_auth.py	2018-05-14 12:42:14 UTC (rev 10282)
@@ -4,7 +4,7 @@
 ##
 ##  See the INN Python Filtering and Authentication Hooks documentation
 ##  for more information.
-##  The perl_auth: parameter in readers.conf is used to load this script.
+##  The python_auth: parameter in readers.conf is used to load this script.
 ##
 ##  An instance of AUTH class is passed to nnrpd via the set_auth_hook()
 ##  function imported from nnrpd.  The following methods of that class
@@ -28,8 +28,8 @@
 ##                                your state variables or close a database
 ##                                connection.  May be omitted.
 ##
-##  If there is a problem with return codes from any of these methods, then nnrpd
-##  will die and syslog the exact reason.
+##  If there is a problem with return codes from any of these methods,
+##  then nnrpd will die and syslog the exact reason.
 ##
 ##  There are also a few Python functions defined in nnrpd:
 ##
@@ -63,22 +63,32 @@
     def authenticate(self, attributes):
         """Called when python_auth: is encountered in readers.conf."""
 
-        # Just for debugging purposes.
-        syslog('notice', 'n_a authenticate() invoked: hostname %s, ipaddress %s, interface %s, user %s' % ( \
-                attributes['hostname'], \
-                attributes['ipaddress'], \
-                attributes['interface'], \
-                attributes['user']))
+        # Just for debugging purposes (in Python 3.x syntax).
+        # By default, do not log passwords (available in attributes['pass']).
+        # syslog('notice', 'n_a authenticate() invoked: hostname %s, ipaddress %s, port %lu, interface %s, intipaddr %s, intport %lu, user %s' % ( \
+        #        attributes['hostname'].tobytes(), \
+        #        attributes['ipaddress'].tobytes(), \
+        #        attributes['port'], \
+        #        attributes['interface'].tobytes(), \
+        #        attributes['intipaddr'].tobytes(), \
+        #        attributes['intport'], \
+        #        (attributes['user'].tobytes() if attributes['user'] else "-")))
 
         # Do username password authentication.
-        if 'foo' == str(attributes['user']) \
-          and 'foo' == str(attributes['pass']):
-            syslog('notice', 'authentication by username succeeded')
-            return (self.authcodes['ALLOWED'], 'No error', 'default_user')
-        else:
-            syslog('notice', 'authentication by username failed')
-            return (self.authcodes['DENIED'], 'Access Denied!')
+        # Python 2.x syntax:
+        #  if attributes['user'] and attributes['pass'] \
+        #    and 'foo' == str(attributes['user']) \
+        #    and 'foo' == str(attributes['pass']):
+        # Python 3.x syntax:
+        #  if attributes['user'] and attributes['pass'] \
+        #    and b'foo' == attributes['user'].tobytes() \
+        #    and b'foo' == attributes['pass'].tobytes():
+        #      syslog('notice', 'authentication by username succeeded')
+        #      return (self.authcodes['ALLOWED'], 'No error', 'default_user')
 
+        syslog('notice', 'authentication by username failed')
+        return (self.authcodes['DENIED'], 'Access Denied!')
+
     def authen_close(self):
         """Called on nnrpd termination."""
         pass
@@ -96,9 +106,10 @@
 
 ##  ...and try to hook up on nnrpd.  This would make auth object methods visible
 ##  to nnrpd.
+import sys
 try:
     set_auth_hook(myauth)
     syslog('notice', "authentication module successfully hooked into nnrpd")
-except Exception, errmsg:    # Syntax for Python 2.x.
-#except Exception as errmsg: # Syntax for Python 3.x.
-    syslog('error', "Cannot obtain nnrpd hook for authentication method: %s" % errmsg[0])
+except Exception: # Syntax valid in both Python 2.x and 3.x.
+    e = sys.exc_info()[1]
+    syslog('error', "Cannot obtain nnrpd hook for authentication method: %s" % e.args[0])

Modified: samples/nnrpd_auth_wrapper.py
===================================================================
--- samples/nnrpd_auth_wrapper.py	2018-05-14 12:33:11 UTC (rev 10281)
+++ samples/nnrpd_auth_wrapper.py	2018-05-14 12:42:14 UTC (rev 10282)
@@ -35,13 +35,17 @@
         # Python 3.x uses memoryview(b'authinfo') because buffers
         # do not exist any longer.  Note that the argument is
         # a bytes object.
-        # attributes['type'] = memoryview(b'authinfo')
-        attributes['type'] = buffer('authinfo')
-        perm = (self.old).authenticate(attributes)
+        #  attributes['type'] = memoryview(b'authinfo')
+        #  perm = (self.old).authenticate(attributes)
+        # whereas in Python 2.x:
+        #  attributes['type'] = buffer('authinfo')
+        #  perm = (self.old).authenticate(attributes)
+        response = 281
         err_str = "No error"
-        if perm[0] == 481:
-            err_str = "Python authentication error!"
-        return (perm[0], err_str)
+        #  if perm[0] == 481:
+        #      response = perm[0]
+        #      err_str = "Python authentication error!"
+        return (response, err_str)
 
     def authen_close(self):
         (self.old).close()
@@ -59,9 +63,10 @@
 
 ##  ...and try to hook up on nnrpd.  This would make auth object methods visible
 ##  to nnrpd.
+import sys
 try:
     set_auth_hook(myauth)
     syslog('notice', "authentication module successfully hooked into nnrpd")
-except Exception, errmsg:    # Syntax for Python 2.x.
-#except Exception as errmsg: # Syntax for Python 3.x.
-    syslog('error', "Cannot obtain nnrpd hook for authentication method: %s" % errmsg[0])
+except Exception: # Syntax valid in both Python 2.x and 3.x.
+    e = sys.exc_info()[1]
+    syslog('error', "Cannot obtain nnrpd hook for authentication method: %s" % e.args[0])

Modified: samples/nnrpd_dynamic.py
===================================================================
--- samples/nnrpd_dynamic.py	2018-05-14 12:33:11 UTC (rev 10281)
+++ samples/nnrpd_dynamic.py	2018-05-14 12:42:14 UTC (rev 10282)
@@ -4,7 +4,7 @@
 ##
 ##  See the INN Python Filtering and Authentication Hooks documentation
 ##  for more information.
-##  The perl_dynamic: parameter in readers.conf is used to load this script.
+##  The python_dynamic: parameter in readers.conf is used to load this script.
 ##
 ##  An instance of DYNACCESS class is passed to nnrpd via the set_auth_hook()
 ##  function imported from nnrpd.  The following methods of that class
@@ -20,13 +20,13 @@
 ##                                newsgroup.  Returns None to grant
 ##                                access, or a non-empty string (which
 ##                                will be reported back to reader)
-##                                otherwise.
+##                                encoded in UTF-8 otherwise.
 ##  dynamic_close()             - Called on nnrpd termination.  Save
 ##                                your state variables or close a database
 ##                                connection.  May be omitted.
 ##
-##  If there is a problem with return codes from any of these methods, then nnrpd
-##  will die and syslog the exact reason.
+##  If there is a problem with return codes from any of these methods,
+##  then nnrpd will die and syslog the exact reason.
 ##
 ##  There are also a few Python functions defined in nnrpd:
 ##
@@ -56,24 +56,32 @@
            readers.conf and a reader requests either read or post
            permission for particular newsgroup."""
 
-        # Just for debugging purposes.
-        syslog('notice', 'n_a dynamic() invoked against type %s, hostname %s, ipaddress %s, interface %s, user %s' % ( \
-                attributes['type'], \
-                attributes['hostname'], \
-                attributes['ipaddress'], \
-                attributes['interface'], \
-                attributes['user']))
+        # Just for debugging purposes (in Python 3.x syntax).
+        # syslog('notice', 'n_a dynamic() invoked: type %s, newsgroup %s, hostname %s, ipaddress %s, port %lu, interface %s, intipaddr %s, intport %lu, user %s' % ( \
+        #        attributes['type'].tobytes(), \
+        #        attributes['newsgroup'].tobytes(), \
+        #        attributes['hostname'].tobytes(), \
+        #        attributes['ipaddress'].tobytes(), \
+        #        attributes['port'], \
+        #        attributes['interface'].tobytes(), \
+        #        attributes['intipaddr'].tobytes(), \
+        #        attributes['intport'], \
+        #        (attributes['user'].tobytes() if attributes['user'] else "-")))
 
         # Allow reading of any newsgroup but not posting.
-        if 'post' == str(attributes['type']):
-            syslog('notice', 'dynamic authorization access for post access denied')
-            return "no posting for you"
-        elif 'read' == str(attributes['type']):
-            syslog('notice', 'dynamic authorization access for read access granted')
-            return None
-        else:
-            syslog('notice', 'dynamic authorization access type is not known: %s' % attributes['type'])
-            return "Internal error";
+        # Python 2.x syntax:
+        #  if 'post' == str(attributes['type']):
+        # Python 3.x syntax:
+        #  if b'post' == attributes['type'].tobytes():
+        #      syslog('notice', 'dynamic authorization access for post access denied')
+        #      return "no posting for you"
+        #  elif 'read' == str(attributes['type']):
+        #      syslog('notice', 'dynamic authorization access for read access granted')
+        #      return None
+        #  else:
+        #      syslog('notice', 'dynamic authorization access type is not known: %s' % attributes['type'])
+        #      return "Internal error";
+        return None
 
     def dynamic_close(self):
         """Called on nnrpd termination."""
@@ -80,8 +88,8 @@
         pass
 
 
-##  The rest is used to hook up the dynamic access module on nnrpd.  It is unlikely
-##  you will ever need to modify this.
+##  The rest is used to hook up the dynamic access module on nnrpd.
+##  It is unlikely you will ever need to modify this.
 
 ##  Import functions exposed by nnrpd.  This import must succeed, or nothing
 ##  will work!
@@ -92,9 +100,10 @@
 
 ##  ...and try to hook up on nnrpd.  This would make auth object methods visible
 ##  to nnrpd.
+import sys
 try:
     set_auth_hook(mydynaccess)
     syslog('notice', "dynamic access module successfully hooked into nnrpd")
-except Exception, errmsg:    # Syntax for Python 2.x.
-#except Exception as errmsg: # Syntax for Python 3.x.
-    syslog('error', "Cannot obtain nnrpd hook for dynamic access method: %s" % errmsg[0])
+except Exception: # Syntax valid in both Python 2.x and 3.x.
+    e = sys.exc_info()[1]
+    syslog('error', "Cannot obtain nnrpd hook for dynamic access method: %s" % e.args[0])

Modified: samples/nnrpd_dynamic_wrapper.py
===================================================================
--- samples/nnrpd_dynamic_wrapper.py	2018-05-14 12:33:11 UTC (rev 10281)
+++ samples/nnrpd_dynamic_wrapper.py	2018-05-14 12:42:14 UTC (rev 10282)
@@ -38,8 +38,8 @@
         (self.old).close()
 
 
-##  The rest is used to hook up the dynamic access module on nnrpd.  It is unlikely
-##  you will ever need to modify this.
+##  The rest is used to hook up the dynamic access module on nnrpd.
+##  It is unlikely you will ever need to modify this.
 
 ##  Import functions exposed by nnrpd.  This import must succeed, or nothing
 ##  will work!
@@ -50,9 +50,10 @@
 
 ##  ...and try to hook up on nnrpd.  This would make auth object methods visible
 ##  to nnrpd.
+import sys
 try:
     set_auth_hook(mydynaccess)
     syslog('notice', "dynamic access module successfully hooked into nnrpd")
-except Exception, errmsg:    # Syntax for Python 2.x.
-#except Exception as errmsg: # Syntax for Python 3.x.
-    syslog('error', "Cannot obtain nnrpd hook for dynamic access method: %s" % errmsg[0])
+except Exception: # Syntax valid in both Python 2.x and 3.x.
+    e = sys.exc_info()[1]
+    syslog('error', "Cannot obtain nnrpd hook for dynamic access method: %s" % e.args[0])



More information about the inn-committers mailing list