[Pkg-freeipa-devel] [Git][freeipa-team/389-ds-base][master] 719 commits: Bugzilla bug 282610: contribute the Netscape svrcore library to open

Timo Aaltonen gitlab at salsa.debian.org
Wed Aug 22 22:49:46 BST 2018


Timo Aaltonen pushed to branch master at FreeIPA packaging / 389-ds-base


Commits:
c15d1ec1 by wtchang%redhat.com at 2005-02-28T23:47:55Z
Bugzilla bug 282610: contribute the Netscape svrcore library to open
source.  This library is used by the LDAP command-line tools and some
Netscape servers.
Added Files: Makefile alt.c cache.c errors.c file.c manifest.mn ntgetpin.c
        pin.c pk11.c std.c svrcore.h user.c

- - - - -
bfa77aae by wtchang%redhat.com at 2005-03-01T00:27:38Z
Fixed text file line endings.
Modified Files: Makefile alt.c cache.c errors.c file.c manifest.mn
        ntgetpin.c pin.c pk11.c std.c svrcore.h user.c

- - - - -
6c9b6426 by wtchang%redhat.com at 2005-03-02T02:29:38Z
I believe we need to define RELEASE (to svrcore) because we define RELEASE
in the top-level manifest.mn files in nss and dbm.

- - - - -
cb14f8d8 by wtchang%redhat.com at 2005-03-02T02:56:41Z
Bugzilla Bug 282610: added the files used in conjunction with ntgetpin.c.
Modified Files: Makefile
Added Files: key.ico logo.ico ntgetpin.rc ntresource.h

- - - - -
9880ac54 by wtchang%redhat.com at 2005-03-02T02:57:47Z
Bugzilla Bug 282610: added a README file.  Describe what svrcore is and
how it is being used.

- - - - -
12df5287 by wtchang%redhat.com at 2005-03-08T17:37:07Z
Upgraded to NSS 3.9.6.

- - - - -
f70643fb by wtchang%redhat.com at 2005-03-08T18:52:50Z
Comment formatting changes.
Modified Files: alt.c cache.c errors.c file.c manifest.mn ntgetpin.c pin.c
                pk11.c std.c svrcore.h user.c

- - - - -
88447d16 by wtchang%redhat.com at 2005-03-09T01:16:42Z
Import NSS 3.9.3 instead.  We canceled the plan to release NSS 3.9.6 this
week.

- - - - -
22875e57 by wtchang%redhat.com at 2005-11-16T01:46:01Z
Upgraded to NSPR 4.6 and NSS 3.10.2.

- - - - -
886581f3 by richm%stanfordalumni.org at 2006-01-06T18:02:35Z
added .spec file for building RPM - package is svrcore-devel - a script svrcore-config for getting cflag and ldflag information, and a .pc file for pkg-config

- - - - -
d3d85838 by richm%stanfordalumni.org at 2006-01-12T00:49:35Z
remove svrcore-config

- - - - -
d8165344 by richm%stanfordalumni.org at 2006-02-01T23:39:01Z
added Windows build instructions - fixed spec file - added patch file for broken coreconf location.mk

- - - - -
2fa85dee by richm%stanfordalumni.org at 2006-05-11T14:51:02Z
Bug: 334561
Description: Clean up spec file for RPM build
Fix Description: Make the spec files use the conventions used by the Fedora/Red Hat packaging system

- - - - -
db0ed858 by richm%stanfordalumni.org at 2006-05-24T13:40:09Z
fixes from Nathan to fix the rpm build script

- - - - -
5ab8d3be by richm%stanfordalumni.org at 2006-06-22T19:38:24Z
bump version to 4.0.2; now using HEAD of mozilla/security/coreconf which has fixed the coreconf location patch, so get rid of patch file

- - - - -
b28f5bbf by richm%stanfordalumni.org at 2006-06-22T19:42:14Z
remove patch file copying

- - - - -
5f09b899 by richm%stanfordalumni.org at 2006-06-23T15:15:07Z
added LICENSE file; renamed svrcore.spec to svrcore-devel.spec and made other changes to comply with fedora packaging guidelines - see https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=196393

- - - - -
4e8d94ae by richm%stanfordalumni.org at 2006-06-23T15:28:29Z
do not erase source tarball

- - - - -
54cb5c0c by richm%stanfordalumni.org at 2006-06-26T15:16:39Z
bump spec revision to 2

- - - - -
1abb01b0 by richm%stanfordalumni.org at 2006-07-13T18:10:21Z
Bump spec rev to 3
Remove unneeded buildrequires perl, gawk, sed
Remove leading / from path macros
Remove provides for package name - done automatically
Move pkgconfig file stuff under install
Added LICENSE and README under docs

- - - - -
358afb2b by gerv%gerv.net at 2006-12-11T09:45:41Z
Bug 236613: change to MPL/LGPL/GPL tri-license.

- - - - -
4c2f1bae by richm%stanfordalumni.org at 2006-12-14T17:07:32Z
Resolves: bug 363168
Description: Add autotool support to svrcore; have svrcore build shared libs
Fix Description: This is a pretty basic autotool-ization using libtool
to build the shared library.  The vast majority of the work was done
by Toshio-san.  However, there are a couple of extras worth noting:
1) Windows builds are different now.  In order to build on Windows using
the MSVC compiler, you still have to use coreconf.  I just could not get
configure/libtool to work correctly with the MSVC compiler.  The old
makefile has been moved to src/Makefile.win, so you just have to
 cd src ; make -f Makefile.win
I also updated the README and INSTALL.win to add this information.
2) I added some m4 files to find and figure out which nspr and nss to use
based on the environment and pkg-config.

- - - - -
db13dd4c by richm%stanfordalumni.org at 2006-12-14T17:09:17Z
Resolves: bug 299995
Description: Use PK11_TokenKeyGenWithFlags instead of PK11_KeyGen
Fix Description: Use the new function and pass in CKF_ENCRYPT|CKF_DECRYPT
as the keygen flags.

- - - - -
7ee36812 by richm%stanfordalumni.org at 2007-03-13T15:39:58Z
Make sure all copyrights/licenses are the mpl/gpl/lgpl tri-license
Update version to 4.0.4
Remove the build rpm script

- - - - -
c5599fad by Noriko Hosoi at 2016-04-07T00:06:37Z
ADD README file.

- - - - -
a0ae37a7 by William Brown at 2016-04-07T01:12:36Z
Merge branch 'master' of ssh://pagure.io/svrcore

- - - - -
fc21da39 by William Brown at 2016-04-07T03:27:52Z
Ticket 389ds #48450 - Implement systemd password support for svrcore

Bug Description:  Svrcore is a secure password collection system. Is is used by
389 Directory Server to collect a pin from a text file, or if not avaliable,
it will ask the user. However, during system start up a tty is not always
avaliable to the user.

Fix Description:  This implement support for systemd's ask-password api as a
client. Additionally, we implement detection of tty connection to the user
module to help improve this. As a result, when a binary is run on the command
line, it will prompt using existing methods. If the tty is not avaliable, it
will contact systemd, which logs a wall message for the user to input a password

If the service is started from systemd, it will be integrated to the startup
command, IE:

$ systemctl start test-interactive.service
Enter PIN for internal (software): *****

https://fedorahosted.org/389/ticket/48450

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
c0cdb1aa by William Brown at 2016-04-07T03:27:58Z
Ticket 48450 - Updates to configure and autotools

Bug Description:  As part of the update to systemd support, we have updated the
configure script and other autotools scripts.

https://fedorahosted.org/389/ticket/48450

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
cf65c356 by William Brown at 2016-04-07T03:38:38Z
Ticket 48450 - Add example driver for svrcore.

Bug Description:  There was previously no example or driver code for svrcore.

Fix Description:  Add an example driver that is able to use the system and
std combined pin types.

https://fedorahosted.org/389/ticket/48450

Author: wibrown

- - - - -
1731600c by William Brown at 2016-04-07T03:42:41Z
SVRCORE issue 1 - Update svrcore license

Bug Description:  The svrcore core code was previously mpl tri license.

Fix Description:  Update to mpl 2.0. Add Red Hat to copyright.

https://pagure.io/svrcore/issue/1

Author: wibrown

- - - - -
73fc7604 by William Brown at 2016-04-08T01:32:01Z
Ticket 3 - headers contain ifdef blocks that should be removed.

Bug Description:  Using ifdef with_systemd in the header is not very clean
as it adds a dependency on clients to know about this.

Fix Description:  Move the ifdef into the systemd functions. The ifdef was
changed to HAVE_SYSTEMD. If not HAVE_SYSTEMD, functions return an error 10,
SVRCORE_MissingFeature, to indicate the feature was not compiled correctly.

https://pagure.io/svrcore/issue/3

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
e7152719 by William Brown at 2016-04-08T01:48:46Z
Update svrcore tools to make releases easier to conduct.

Release version 4.1.0

Author: wibrown

- - - - -
5df6f4b0 by William Brown at 2016-04-13T22:32:42Z
Ticket 5 - Integrate asan support for code quality checking

Bug Description:  Coverity found a number of defects that could have been
detected with asan use during testing.

Fix Description:  Include the --enable-asan flag to make it easier to run with
stricter checks during development.

https://pagure.io/svrcore/issue/5

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
ecc39933 by William Brown at 2016-04-13T22:32:42Z
Ticket 10 - Use after free

Bug Description:  Coverity and ASAN detected use after frees related to the
reuse of the pin object.

Fix Description:  This corrects the behaviours that would cause the use
after free to occur

https://pagure.io/svrcore/issue/10

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
da5bbc5c by William Brown at 2016-04-13T22:32:42Z
Ticket 7 - Incorrect result check

Bug Description:  Coverity detected an issue where tmp_fd was not checked for
null with fopen.

Fix Description:  Add the correct checks for fopen
Add checks for malloc return codes.

https://pagure.io/svrcore/issue/7

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
c52cb926 by William Brown at 2016-04-13T22:32:42Z
Ticket 6 - Resource leak in systemd ask pass

Bug Description:  In an error case, systemd would set the pin to "", which would
leak the token memory.

Fix Description:  strncpy the "" into the token to prevent the leak.

https://pagure.io/svrcore/issue/6

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
e248c0c5 by William Brown at 2016-04-13T22:32:42Z
Ticket 8 - Coverity compiler warnings

Bug Description:  Coverity detected a number of compiler warnings.

Fix Description:  Correct the issues in format strings that triggered the
warnings.

https://pagure.io/svrcore/issue/8

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
b594b41e by William Brown at 2016-04-13T22:32:43Z
Ticket 9 - Coverity deadcode

Bug Description:  Coverity detected deadcode in the systemd handling of pins

Fix Description:  Fix the goto statements that accidentally caused the deadcode

Note: This also updates the configure files

https://pagure.io/svrcore/issue/9

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
e1417431 by William Brown at 2016-04-13T22:32:43Z
Ticket 12 - update spec to match fedora 4.1.0

Bug Description:  There was a change between fedora's spec, and ours. Update
the svrcore spec file.

Fix Description:  Update the spec

https://pagure.io/svrcore/issue/12

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
f49c3c40 by William Brown at 2016-04-13T22:43:03Z
Release 4.1.1 of svrcore

 Description: Release 4.1.1 of svrcore.

Author: wibrown

- - - - -
57028d1c by Noriko Hosoi at 2016-04-21T20:16:01Z
Bug 1329002 - SVRCORE - Fixing coverity issues.

Description: Defect type: UNINIT
15. svrcore-4.1.1/src/systemd-ask-pass.c:409: uninit_use_in_call:
    Using uninitialized value "*socket_path" when calling "unlink".
16. svrcore-4.1.1/src/systemd-ask-pass.c:413: uninit_use_in_call:
    Using uninitialized value "*ask_path" when calling "unlink".
17. svrcore-4.1.1/src/systemd-ask-pass.c:418: uninit_use_in_call:
    Using uninitialized value "*tmp_path" when calling "unlink".

- - - - -
2e211a68 by Noriko Hosoi at 2016-04-21T20:30:23Z
Release 4.1.2 of svrcore

Description: Release 4.1.2 of svrcore.
Note: updated autogen.sh to allow using newer automake and autoconf.

- - - - -
ed490d14 by William Brown at 2016-07-18T23:03:39Z
Ticket 14 - svrcore does not detect tty

Bug Description:  Due to a mistake the detection was hardcoded to false

Fix Description:  Fix the check

https://pagure.io/svrcore/issue/14

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
2e8ffb5e by William Brown at 2016-10-04T03:36:29Z
Ticket 16 - with systemd should unset have systemd if pkgconfig not found

Bug Description:  On platforms that don't have systemd, we should check for this
and correctly disable the systemd parts of svrcore.

Fix Description:  Add pkgconfig check to configure that will automatically
disable systemd support if the pc file is not found.

https://pagure.io/svrcore/issue/16

Author: wibrown

Review by: lslebodn

- - - - -
20a4942f by William Brown at 2016-10-06T22:49:27Z
Ticket 17 - update stdc to c99 to match other projects

Bug Description:  In the ds project we are moving to c99 across the board.

Fix Description:  Update svrcore to match

https://pagure.io/svrcore/issue/17

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
cae89035 by William Brown at 2016-10-25T05:05:36Z
Ticket 18 - Remove configure outputs

Bug Description:  Remove the configure outputs just like DS

Fix Description:  for i in `cat .gitignore`; do rm -rf $i; done

https://pagure.io/svrcore/issue/18

Author: wibrown

Review by: nhosoi

- - - - -
bc12b6d5 by Mark Reynolds at 2017-03-10T19:40:51Z
Bump version to 4.1.3

- - - - -
f40c937d by Mark Reynolds at 2017-09-22T15:34:24Z
Bump version to 1.4.0

- - - - -
8fc58fd2 by Mark Reynolds at 2017-09-22T15:38:49Z
Revise VERSION number

- - - - -
6b1cbc8f by Ludwig Krispenz at 2017-09-26T15:57:47Z
Ticket: 49180 - errors log filled with attrlist_replace - attr_replace

Bug: If a RUV contains the same URL with different replica IDs the created referrals contain duplicates

Fix: check duplicate referrals

Reviewed by: Mark, thanks

- - - - -
83f04fe8 by Ludwig Krispenz at 2017-09-27T09:07:47Z
 Ticket: 49180  - add CI test

- - - - -
af723fd6 by Mark Reynolds at 2017-09-27T13:32:48Z
Ticket 49305 - Need to wrap atomic calls

Bug Description:  Some RHEL 7.5 platforms (ppc 32bit) still do not support
                  all the gcc builtin atomics.  This breaks the downstream
                  builds.

Fix Description:  Use wrapper functions for the atomic's using #define's
                  to detect if builtin atomics are supported, otherwise
                  use the egneric nspr atomic functions.

https://pagure.io/389-ds-base/issue/49305

Reviewed by: tbordaz(Thanks!)

- - - - -
e9ad5f5a by William Brown at 2017-09-28T03:25:55Z
Ticket 49378 server init fails

Bug Description:  We used our own target for DS installation, but
we should just use multi-user like anything else.

Fix Description:  Change service template to multi-user. This should
be a seamless upgrade to most consumers.

https://pagure.io/389-ds-base/issue/49378

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
1d158cd0 by Thierry Bordaz at 2017-09-28T09:19:45Z
Ticket 48973 Indexing a ExactIA5Match attribute with a IgnoreIA5Match matching rule triggers a warning

Bug Description:
	When a index configuration entry (i.e. "cn=<attr>,cn=index,cn=<be_name>,..") contains
	one or more nsMatchingRule values.
	A weird warning message can be logged.

	This message should not be a warning, it is more a debug message.
	Indeed it exists two mechanisms to register MR. So to retrieve an approriate MR
	the server is first using the old mechanism and if it fails it uses the new one.
	The old mechanism was going throught the set of indexer function checking one that supports the
	MR specified in nsMatchingRule.
	Usually indexer function are not defined and it fallback to new mechanism.
	In case of several nsMatchingRule, an indexer function is assigned to an MR.

	It is normal that a given MR does not manage syntax it is not designed for.
	so the message should be DEBUG.

	The message is not that frequent because using nsMatchingRule different from the
	attribute syntax is not frequent.

	This erronous message came from https://fedorahosted.org/389/ticket/48745

Fix Description:
	Switch message from warning to debug

https://pagure.io/389-ds-base/issue/48973

Reviewed by: Mark Reynolds and William Brown (thanks !)

Platforms tested: F23

Flag Day: no

Doc impact: no

- - - - -
93a29584 by Mark Reynolds at 2017-09-29T19:13:15Z
Ticket 49305 - Need to wrap atomic calls

Bug Description:  Some RHEL 7.5 platforms (ppc 32bit) still do not support
                  all the gcc built-in atomics.  This breaks the downstream
                  builds.

Fix Description:  Use wrapper functions for the atomic's using #define's
                  to detect if builtin atomics are supported, otherwise
                  use the generic nspr atomic functions.

https://pagure.io/389-ds-base/issue/49305

Reviewed by: tbordaz, lkrispen, and wibrown(Thanks!!!)

- - - - -
805e8f4d by Mark Reynolds at 2017-10-02T14:11:10Z
Ticket 49385 - Fix coverity warnings

Description:  This fixes coverity issues found from RHEL build of 1.3.7

https://pagure.io/389-ds-base/issue/49385

Reviewed by: lkrispenz(Thanks!)

- - - - -
ee25b881 by William Brown at 2017-10-03T00:40:05Z
Ticket 49387 - pbkdf2 settings were too aggressive

Bug Description:  Our initial settings were too aggresive and caused some
cpu latency issues. We should tone these down a bit, and then step
them up slower.

Fix Description:  Decrease the test rounds at start up, lower the minimum
to 2048, and decrease the time factor to 4 ms rather than 40.

Cleanup to int types.

https://pagure.io/389-ds-base/issue/49387

Author: wibrown

Review by: mreynolds (Thanks mate!)

- - - - -
4b41a024 by Mark Reynolds at 2017-10-03T12:18:36Z
Ticket 49388 - repl-monitor - matches null string many times in regex

Bug Description:  When using a wildcard(*) for the hostname, some of the
                  regex's for parsing the various configurations throws
                  out warnings.

Fix Description:  When a wildcard is detected reset the hostnode variable
                  to nothing.

https://pagure.io/389-ds-base/issue/49388

Reviewed by: firstyear(Thanks!)

- - - - -
a6d2c684 by Mark Reynolds at 2017-10-03T12:30:33Z
Ticket 49092 - Add CI test for schema-reload

Description: ADD CI test for schema reload task

https://pagure.io/389-ds-base/issue/49092

Reviewed by: spichugi & firstyear(Thanks!!)

- - - - -
0953e601 by Mark Reynolds at 2017-10-04T12:13:02Z
Ticket 49389 - unable to retrieve specific cosAttribute when subtree
               password policy is configured

Bug Description:   If indirect cos is being used and a subtree password
                   policy is added, th orignal COS attributes aren't always
                   returned.  The issue is that when the subtree password
                   policy attribute was encountered during the virtual
                   attribute processing it set a flag that said the attribute
                   was operational (which is correct for the password policy
                   attr: pwdpolicysubentry).

                   However, this flag was accidentally carried over to the
                   following virtual attributes that were being processed.
                   Which caused those attributes to be seen as operational
                   which is why it was no longer being returned to the client.

Fix Description:   Reset the prop flags before processing the next COS attribute

https://pagure.io/389-ds-base/issue/49389

Reviewed by: firstyear(Thanks!)

- - - - -
40608484 by Mark Reynolds at 2017-10-04T13:27:04Z
Ticket 49320 - Activating already active role returns error 16

Bug Description:  ns-activate.pl returns error 16 when trying to activate an
                  already active role.

Fix Description:  Check for error 16 (no such attr), and return error 100.
                  Also added a "redirect"otion to the ldapmod function to
                  hide any errors printed to STDERR, so that the script can
                  display its own error message.

https://pagure.io/389-ds-base/issue/49320

Reviewed by: firstyear(Thanks!)

- - - - -
4cd1a24b by William Brown at 2017-10-04T23:08:12Z
Ticket 49372 - filter optimisation improvements for common queries

Bug Description:  Due to the way we apply indexes to searches
and the presence of the "filter test threshold" there are a number
of queries which can be made faster if they understood the internals
of our idl_set and index mechanisms. However, instead of expecting
application authors to do this, we should provide it.

Fix Description:  In the server we have some cases we want to
achieve, and some to avoid:

* If a union has an unindexed candidate, we throw away all work
  and return an ALLIDS idls.
* In an intersection, if we have an idl that is less than
  filter test threshold, we return immediately that idl
  rather than accessing all others, and perform a filter
  test.

Knowing these two properties, we can now look at improving filters
for queries.

In a common case, SSSD will give us a query which is a union of
host cn and sudoHost rules. However, the sudoHost rules are
substring searchs that are not able to be indexed - thus the whole
filter becomes an unindexed search. IE:

(|(cn=a)(cn=b)(cn= ....)(sudoHost=[*]*))

So in this case we want to move the substring to the first query
so that if it's un-indexed, we fail immediately with ALLIDS rather
than opening the cn index.

For intersection, we often see:

(&(objectClass=account)(objectClass=posixAccount)(uid=william))

The issue here is that the idls for account and posixAccount both
may contain 100,000 items. Even with idl lookthrough limits, until
we start to read these, we don't know if we will exceed that.

A better query is:

(&(uid=william)(objectClass=account)(objectClass=posixAccount))

Because the uid=william index will contain a single item, this
put's us below filter test threshold, and we will not open the
objectClass indexes.

In fact, in an intersection, it is almost always better to perform
simple equalities first:

(&(uid=william)(modifyTimestamp>=...)(sn=br*)(objectClass=posixAccount))

In most other cases, we will not greatly benefit from re-arrangement
due to the size of the idls involved we won't hit filter test. IE

(&(modifyTimestamp>=...)(sn=br*)(objectClass=posixAccount))

Would not be significantly better despite and possible arrangement
without knowing the content of sn.

So in summary, our rules for improving queries are:

* unions-with-substrings should have substrings *first*
* intersection-with-equality should have all non-objectclass
  equality filters *first*.

https://pagure.io/389-ds-base/issue/49372

Author: wibrown

Review by: lkrispen, mreynolds (Thanks!)

- - - - -
1ff5f4a4 by William Brown at 2017-10-05T21:54:45Z
Ticket 49279 - remove dsktune

Bug Description:  dsktune is a difficult to maintain and now
defunct part of our suite. Due to our shipping proper tuned
system defaults, we no longer require this. Most of it's functions
are related to solaris patching from the 90's

Fix Description:  rm idsktune.c

https://pagure.io/389-ds-base/issue/49279

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
816ffee3 by William Brown at 2017-10-05T21:55:21Z
Ticket 49235 - pbkdf2 by default

Bug Description:  Now that we have improved the tuning somewhat
we should offer a stronger default password hash. Wehave historically
been poor at offering secure defaults, so this is a strong move
to support this.

Fix Description:  PBKDF2 by default. PBKDF2 helps to prevent
certain classes of attacks by being resistant to bruetforce
attacks due to the high work factor that an attacker must
conduct to attempt to check the passwords content. Additionally
the PBKDF2 impl we provide has a high random salt content (well
in excess of current NIST requirements) which assits making
bruteforces harder.

https://pagure.io/389-ds-base/issue/49235

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
0ea523c0 by William Brown at 2017-10-05T21:56:02Z
Ticket 49392 - memavailable not available

Bug Description:  On certain linux platforms memAvailable is
not actually available! This means that the value was 0, so
cgroup max was read instead, setting the system ram to:

9223372036854771712

That's a bit excessive, and can cause memory allocations to fail.

Fix Description:  If memavail can't be found, fall back to
memtotal instead.

https://pagure.io/389-ds-base/issue/49392

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
1fe2c761 by Mark Reynolds at 2017-10-06T15:13:50Z
Ticket 49038 - remove legacy replication - change cleanup script precedence

Description:  Bump the cleanup scripts precendance so it happens after the
              main plugin upgrade scripts are called.

https://pagure.io/389-ds-base/issue/49038

Reviewed by: firstyear(Thanks!)

- - - - -
22e54fac by Mark Reynolds at 2017-10-09T14:10:40Z
Bump version to 1.4.0.1

- - - - -
4929d347 by Nathan Kinder at 2017-10-11T15:24:03Z
Add README file

- - - - -
bf919d1d by Jan Rusnacko at 2017-10-11T15:24:03Z
Initial commit with DSModuleProxy.

In order to keep implementation of class DSInstance organized, we
implement all functionality as functions in separate modules. All
functions accept DSInstance object as their first argument by rule.
Even though it`s beneficial to define everythin as function in
separate module, in runtime it`s nicer to have methods on DSInstance.

ModuleProxy serves two purposes:
 - takes modules and all functions within and adds them to DSInstance
   as methods
 - creates separate namespaces for all modules within DSInstance

- - - - -
2591bd60 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:03Z
Ticket ticket47566 - Initial import of DSadmin into 389-test repos

Bug Description:

	This commit is done with a merge of https://github.com/richm/dsadmin/pull/5
	that is a refactoring of dsadmin (introducing replication/backend/replica/suffix)
	At the time of this commit https://github.com/richm/dsadmin/pull/5 was not yet push in
	master https://github.com/richm/dsadmin

	Changes regarding https://github.com/richm/dsadmin are:
		- removal of instance
		- support local host name different than localhost
		- renaming of dsadmin/dsadmin into lib389/lib389
		- support of sudoers

https://fedorahosted.org/389/ticket/47566

Reviewed by: Rich Megginson (thanks Rich for the review)

Platforms tested: <plat>

Flag Day: no

Doc impact: no

- - - - -
943ff787 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:03Z
Ticket 47568 - Rename DSAdmin class

Bug Description:
	This bug is to rename DSAdmin main class with an other name: DirSrv
	This change impact the lib389 as well as the tests.

Fix Description:
        - Rename
                directory: 'dsadmin' -> 'lib389'
                class    : 'DSAdmin' -> 'DirSrv'
                object   : 'dsadmin' -> 'dirsrv'
        - Add Index class, with method to delete indexes of a backend
          (for backend deletion)
        - Add 'delete' method in class backend
        - To run as regular user (/usr/bin/sudo, select ports > 1024)
        - Replace addbackend_harn per add backend+suffix
	- Fix various unit tests (almost all of them are pass)
	- Use local variable naming convention: camel_case

https://fedorahosted.org/389/ticket/47568

Reviewed by: Rich Megginson, Roberto (thanks you both for the review)

Platforms tested: F17

Flag Day: no

Doc impact: no

- - - - -
3344b43b by Thierry bordaz (tbordaz) at 2017-10-11T15:24:03Z
Ticket 47578: CI tests: removal of 'sudo' and absolute path in lib389

Bug Description:
	During the first drop of dsadmin into lib389 I added sudo call
	and absolute path

	On platform that do not support SELinux, failure to call 'semanage' is
	not caught

Fix Description:
	suppress the call to sudo and catch failure of 'semanage' call

https://fedorahosted.org/389/ticket/47578

Reviewed by: Rich Megginson (Thanks Rich !)

Platforms tested: F17

Flag Day: no

Doc impact: no

- - - - -
b5294b46 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:03Z
Ticket 47584: CI tests: add backup/restore of an instance

Bug Description:
     A test case or a test suite needs to be isolated from side effects of others
     test cases run previously. For this we need to run a test case with
     "clean" instances.

Fix Description:
     Creating/deleting instances is consuming time and could take much more
     time than the test case itself.

     The idea, is to create a kind of instance backup file. This backup file
     contains all components of the instance: config, schema, database
     environement, database files, certificates/keys...

     Before running a test case, the test reinit the instance it needs from
     this backup. So it gets rapidely  "clean" instances

https://fedorahosted.org/389/ticket/47584

Reviewed by: Rich Megginson, Roberto Polli (Thank you both for your help/patience !)

Platforms tested: F19

Flag Day: no

Doc impact: no

- - - - -
deeb34e1 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:03Z
Ticket 47590: CI tests: add/split functions around replication

Bug Description:
	Functions to setup replication are a bit too complex. They do a lot of things that
	should be split in several simpler functions.
	The parameters are also complex with mandatory/optional.

Fix Description:
	The fix implements:
		- delSchema(attr, val)
		- getSchemaCSN(instance)
		- New class Agreement (in brooker)
			- status
			- schedule
			- create
			- init
			- checkProperties
			- _check_interval
		- reorganise routine from lib389 and brooker.replica to brooker.Agreement
		  (init, status, create, schedule)
		- Creation of a default replica Mgr moved from lib389
		  to brooker.replica.create_repl_manager
		- Add a fix into instancebackupFS to backup changelog directory

https://fedorahosted.org/389/ticket/47590

Reviewed by: Roberto Polli, Rich Megginson (Many thanks for all your Help Roberto and Rich)

Flag Day: no

Doc impact: no

- - - - -
31f875c5 by Rich Megginson at 2017-10-11T15:24:03Z
do not print ERROR message every time creating an instance

Reviewed by: tbordaz (Thanks!)

- - - - -
89e32553 by Rich Megginson at 2017-10-11T15:24:03Z
need time and datetime - add str method for RUV to format RUV in a readable format

Reviewed by: tbordaz (Thanks!)

- - - - -
ff531963 by Rich Megginson at 2017-10-11T15:24:03Z
cannot modify passed in args - make a deepcopy to avoid side effects

Reviewed by: tbordaz (Thanks!)

- - - - -
f52aeb5c by Rich Megginson at 2017-10-11T15:24:03Z
added agreement stop and restart methods

Reviewed by: tbordaz (Thanks!)

- - - - -
1de8c2f9 by Rich Megginson at 2017-10-11T15:24:03Z
need to set correct replica type

Reviewed by: tbordaz (Thanks!)

- - - - -
13cf56fc by Rich Megginson at 2017-10-11T15:24:04Z
add convenience agreement_dn method to get a single replication agreement dn

Reviewed by: tbordaz (Thanks!)

- - - - -
e47d7002 by Rich Megginson at 2017-10-11T15:24:04Z
file did not end with newline

Reviewed by: tbordaz (Thanks!)

- - - - -
016b0117 by Rich Megginson at 2017-10-11T15:24:04Z
move stop and restart to agreement.pause and agreement.unpause

Reviewed by: tbordaz (Thanks!)

- - - - -
a6312f47 by Rich Megginson at 2017-10-11T15:24:04Z
ignore patch files

- - - - -
cf14d5e6 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:04Z
Ticket 47595 - fail to detect/reinit already existing instance/backup

Bug Description:
	We need to re-initialize an instance from a previously taken backup.
	lib389 had not the capability to test if an instance existed and a backup existed.

Fix Description:
	Fix for backup/restore:
		- before restore clean up
			DB directory from previous '.db' files and backend dirs
			errors/access log files
	Testing existance of backup/instance
		- support offline dirsrv instance for offline access
		- add checkInstance/_offlineDirsrv/existsBackup/existsInstance
		- fix clearInstanceBackupFS
	Misc:
		- fix backup/restore functions (replica.changelog, instanceBackupFS, instanceRestoreFS)

https://fedorahosted.org/389/ticket/47595

Reviewed by: Rich Megginson

Platforms tested: Fedora 19 (jenkins)

Flag Day: no

Doc impact: no

- - - - -
ab718daa by Thierry bordaz (tbordaz) at 2017-10-11T15:24:04Z
Change enableReplication/Replica.add interface to use role. + error handling with exception

Reviewed by: Rich Megginson (thank rich for all your reviews)

- - - - -
764acac6 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:04Z
Ticket 47625 - CI lib389: DirSrv not conform to the design

Bug Description:
	Changes to make DirSrv class conform to the design  http://port389.org/wiki/Upstream_test_framework

Fix Description:
	The fix introduces several changes
		- Define SERVER properties with "verb" that could be used in CLI
                  Those properties are used in 'args' dictionary to handle the instance (create/delete...)
		- Split the previous lib389.DirSrv:__init__, __localinit__ and __initPart2 into
                  __init__/allocate/create/open
		- Implements lib389.DirSrv:delete/close/start/stop/restart/list/exists
		- Copy tools functions related to instance backup into instance class lib389.DirSrv
			clearBackupFS/checkBackupFS/backupFS/restoreFS
		- Add a new test module for DirSrv: dirsrv_test.py
		- List returns the properties retrieve from the config files
                  (<prefix>/etc/sysconfig/dirsrv-* or $HOME/.dirsrv/dirsrv-*

https://fedorahosted.org/389/ticket/47625

Reviewed by: Rich Megginson (thanks)

Platforms tested: F17

Flag Day: no

Doc impact: http://port389.org/wiki/Upstream_test_framework

- - - - -
91d16124 by Rich Megginson at 2017-10-11T15:24:04Z
need package name in exception

Reviewed by: nkinder (Thanks!)

- - - - -
a486d016 by Rich Megginson at 2017-10-11T15:24:04Z
add changelog config method

Reviewed by: nkinder (Thanks!)

- - - - -
6b4e0432 by Rich Megginson at 2017-10-11T15:24:04Z
fix deprecated repl args in dirsrv class - use default timeout of 120

Reviewed by: nkinder (Thanks!)

- - - - -
13dd2b61 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:04Z
Ticket 47635: MT/Backend/Suffix to be conform with the design

Bug Description:
	This bug is to create MappingTree/Suffix classes and to move Backend from brooker.py
	to a dedicated file.
	Then to implement the interface of each classes as described in http://port389.org/wiki/Upstream_test_framework

Fix Description:

	Implements:

		Mapping Tree:
			mappingTree.py: MappingTree/list-create-delete-getProperties-setProperties-toSuffix
			mappingTree_test.py: unit tests
		Backend:
			brooker.py: remove Backend from that file
			backend.py: Backend/list-delete-create-getProperties-setProperties-toSuffix
			make readonly local function that will be reimplemented with setProperties
			backend_test.py
			suppress/replace methods implemented in others classes
			  setupBackend/getSuffixForBackend
		Suffix:
			suffix.py:Suffix/list-toBackend-getParent
			suffix_test.py: unit tests
			suppress/replace methods implemented in others classes
			  getSuffixes/setupSuffix/getBackendsForSuffix/findParentSuffix/addSuffix

			  getSuffixes/setupBackend/setupSuffix/getBackendsForSuffix/getSuffixForBackend/findParentSuffix/addSuffix

https://fedorahosted.org/389/ticket/47635

Reviewed by: Rich Megginson

Platforms tested: F17

Flag Day: no

Doc impact: no

- - - - -
e9eff97c by Thierry bordaz (tbordaz) at 2017-10-11T15:24:04Z
Fix for running on RPM (not -- prefix)

Reviewed by: Nathan Kinder (thanks Nathan)

- - - - -
eb2e1760 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:05Z
Make the backup (instance) directory readable by anybody

Reviewed by: Nathan Kinder(thanks Nathan)

- - - - -
8808f1a9 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:05Z
Ticket 47652 - replica add fails: MT.list return a list not an entry

Reviewed by: Nathan Kinder

- - - - -
c412ea6b by Thierry bordaz (tbordaz) at 2017-10-11T15:24:05Z
Ticket  47600 : Replica/Agreement/Changelog not conform to the design

Bug Description:
	Currently all replication functions are in brooker/__ini__.
	To conform the design, we need to split these functions into various classes
	Replica/Agreement/Changelog (and files) and to remove them from brooker/__init__.
	Also to change the interfaces (list/create/delete/set/get..).

Fix Description:
	Create Agreement/Replica/Changelog classes with (list/create/delete/set-getProperties...)
	Add unit tests for Replica and RA
	Fix some issues found with unit tests

https://fedorahosted.org/389/ticket/47600

Reviewed by: Rich Megginson (thanks Rich !!)

Platforms tested: F17

Flag Day: no

Doc impact: no

- - - - -
b28d8b4b by Thierry bordaz (tbordaz) at 2017-10-11T15:24:05Z
Ticket 47671 - CI lib389: allow to open a DirSrv without having to create the instance

Bug Description:
	DSAdmin allowed to bind to an already existing instance without creation of the instance.
	Allocate() and open() require that a server-id is provided during Allocate.
	This prevents to allocate a DirSrv? to bind to an existing instance.

Fix Description:
	Change Allocate so that SER_SERVERID_PROP is not mandatory.
	Change Open so that if self.serverid is not defined it retrieves it from the instance (self.inst)

https://fedorahosted.org/389/ticket/47671

Reviewed by: Rich Megginson

Platforms tested: F17/F19(jenkins)

Flag Day: no

Doc impact: no

- - - - -
ef1fd030 by Rich Megginson at 2017-10-11T15:24:05Z
Ticket #47648 lib389 - add schema classes, methods

https://fedorahosted.org/389/ticket/47648
Reviewed by: tbordaz (Thanks!)
Branch: master
Fix Description: Add a schema.py and a Schema class for use as
dirsrv.schema.  It has methods to add/delete schema elements, to get a list
of schema files for an instance, and to get the schema as an LDAP entry or
as a python-ldap SubSchema object.
Platforms tested: RHEL6 x86_64
Flag Day: no
Doc impact: no

- - - - -
fbc9b868 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:05Z
Ticket 47695 - Add plugins/tasks/Index

Bug Description:
	Add plugins, tasks and index in lib389

Fix Description:
	This creates new modules (index/plugin/tasks).
	(some methods from brooker/__init__.py are moved to these modules)
		tasks: importLDIF,exportLDIF, reindex, fixupMemberOf
		plugins: list, enable, disable
		index: addIndex, modIndex, create (wrapper of addIndex).

https://fedorahosted.org/389/ticket/47695

Reviewed by: Rich Megginson

Platforms tested: F19(jenkins)

Flag Day: no

Doc impact: no

- - - - -
fc73eec4 by Mark Reynolds at 2017-10-11T15:24:05Z
Add all the plugin names to the constants file

Reviewed by: tbordaz(Thanks!)

- - - - -
b98d6949 by Mark Reynolds at 2017-10-11T15:24:05Z
Ticket 47819 - Add the new precise tombstone purging config attribute

Description:  Add REPLICA_PRECISE_PURGING:
                  'nsds5ReplicaPreciseTombstonePurging'

Reviewed by: rmeggins(Thanks!)

- - - - -
fa2361ff by Mark Reynolds at 2017-10-11T15:24:05Z
Ticket 47845 - Add backup/restore/fixup tombstone tasks to lib389

Description:  Add backup/restore/fixup tombstone tasks to lib389.
              Also improved the task time stamp value to be always
              unique.

https://fedorahosted.org/389/ticket/47845

Reviewed by: tbordaz(Thanks!!)

- - - - -
387ea344 by Mark Reynolds at 2017-10-11T15:24:05Z
Ticket 47851 - Add function to retrieve dirsrvtests data directory

Description: In 389-ds-base source a new data directory was created:

                   ds/dirsrvtests/data

             This is used to store LDIF files, backups, etc.  In order
             to access these files, regardless of where the script is
             run from, we need a function to return the correct full
             path to the data directory.

Reviewed by: nhosoi(Thanks!)

- - - - -
8836661c by Mark Reynolds at 2017-10-11T15:24:05Z
Ticket 47845 - add stripcsn option to tombstone fixup task

Description: add the stripcsn option to fixup task, so the task can can
             easily be verified

https://fedorahosted.org/389/ticket/47845

Reviewed by: nhosoi(Thanks!)

- - - - -
523e1a8b by Mark Reynolds at 2017-10-11T15:24:05Z
Ticket 47851 - Need to retrieve tmp directory path

description:  lib389 tets also needs  tmp directory to store
              files created during runtime.  Modify the "get"
              function to be able to return the data or tmp
              directory paths.

https://fedorahosted.org/389/ticket/47851

Reviewed by: nhosoi(Thanks!)

- - - - -
9974af9d by Mark Reynolds at 2017-10-11T15:24:06Z
Ticket 47855 - Add function to clear tmp directory

Reviewed by: ?

- - - - -
e6e0679b by Milan Kubík at 2017-10-11T15:24:06Z
Ticket 47848 : Add support for setuptools.

https://fedorahosted.org/389/ticket/47848
Resolves: Ticket 47848

Bug Description:
lib389 does not support packaging at the moment

Reviewed by: ???

Fix Description:
The patch added setup.py file that uses python's setuptools, making it
a source distribution of the package.
Added setup.cfg contains information needed to build RPM package.

Platforms tested: RHEL 7
Flag Day: no
Doc impact: yes
The script snippets on the project wiki page describing the installation
proces will have to be updated to reflect this change.
Considering the possibility of offering built binary package as well
(rpm, wheel).

- - - - -
cb07ce9b by Thierry bordaz (tbordaz) at 2017-10-11T15:24:06Z
start/stop may hang indefinitely (like SSL init fails)

Description:
	when rerun a start/stop command, check the timeout is not hit

- - - - -
8681a0a4 by Thierry bordaz (tbordaz) at 2017-10-11T15:24:06Z
Ticket 47691 - using lib389 with RPMs

Bug Description:
	Ability to use lib389 as 'root' user with RPM install

Fix Description:
	- Check that '127.0.0.1' is resolved as localhost.localdomain
	- prefix becomes '/' when running as root
	- sysconfig files contains '\n'
	- create a 'nobody' user if it does not exist
	- make sure to backup/restore files and directories as well

https://fedorahosted.org/389/ticket/47691

Reviewed by: Rich (many many thanks for all reviews)

Platforms tested: F17/F20

Flag Day: no

Doc impact: no

- - - - -
0be6d3ea by Mark Reynolds at 2017-10-11T15:24:06Z
Bug Description:  Disable plugins had a copy and paste error that prevented
                  plugins from actaully being disabled.

Fix Description:  When disabling plugins, use the "off" value

- - - - -
550f106d by Mark Reynolds at 2017-10-11T15:24:06Z
Ticket 47990 - Add "upgrade" function to lib389

Bug Description:  There is no existing function to run "upgrades" using
                  lib389.

Fix Description:  Add a new "upgrade" fucntion to lib389.  This function
                  takes a single DirSrv instance, and uses its prefix to
                  upgrade all the servers.

https://fedorahosted.org/389/ticket/47990

Reviewed by: rmeggins(Thanks!)

- - - - -
5bae7736 by Mark Reynolds at 2017-10-11T15:24:06Z
Ticket 47990 - Add check for ".removed" instances when doing an upgrade

Description:  Add check to not process instances names that end with ".removed"

              Also did some code cleanup (removed whitespaces and fixed formatting)

https://fedorahosted.org/389/ticket/47990

Reviewed by: nhosoi(Thanks!)

- - - - -
a994df48 by Mark Reynolds at 2017-10-11T15:24:06Z
Ticket 48000 - replica agreement pause/resume should have a short sleep

Bug Description:  There is a small window between disabling a replica agreement and it actually
                  stopping the agreement thread. This can allow one more replication session
                  to slip through, which can cause issues with lib389 tests. Replication logging
                  can also widen this "window".

Fix Description:  Add a short sleep after pausing/resuming a replication agreement.

https://fedorahosted.org/389/ticket/48000

Reviewed by: rmeggins(Thanks!)

- - - - -
b9c4e541 by Mark Reynolds at 2017-10-11T15:24:06Z
 Ticket 48004 - Fix various issues

Bug Description:  The get_dir() and clearTmpDir() functions do not work with
                  the 389 suite tests.  The 389 constants.py variables should
                  be added to the lib389 _constants.py file

Fix Description:  Make the tmp and data directory logic more robust to handle
                  different test locations.  Moved all the "constants" from
                  389 constants.py to lib389 _constants.py - also updated the
                  replication constants to include more masters, and hubs,
                  consumers, and standalone instances (10 each).

https://fedorahosted.org/389/ticket/48004

Reviewed by: tbordaz(Thanks!!)

- - - - -
19d732ec by Mark Reynolds at 2017-10-11T15:24:06Z
Ticket 48000 - Repl agmts need more time to stop

Bug Description:  The existing sleep of 2 seconds is apparently not enough
                  time for repl agmt thread to stop.

Fix Description:  Extend the sleep to 5 seconds when "pausing" an agreement

https://fedorahosted.org/389/ticket/48000

Reviewed by: nhosoi(Thanks!)

- - - - -
3a4be47f by Mark Reynolds at 2017-10-11T15:24:09Z
Ticket 48020 - lib389 - need to reset args_instance with
 every DirSrv init

Bug Description:  When running multiple tests using py.test, the global
                  dictionary "args_instance" is not reset.  So settings
                  from previous DirSrv instances are applied to the new
                  instance of DirSrv(if not explicity set in the calling
                  test).

Fix Description:  Reset "args_instance" when a DirSrv object is initialized.
                  Also used "PW_DM"  for all the SER_ROOT_PW values.

https://fedorahosted.org/389/ticket/48020

Reviewed by: nhosoi(Thanks!)

- - - - -
2e0ea915 by Mark Reynolds at 2017-10-11T15:24:09Z
Ticket 48023 - create function to test replication between servers

Bug Description:  There is no standard way to test that replication
                  is working between two or more servers.

Fix Description:  Create a function to test replication between replicas.  The
                  function can accept any number of replicas(DirSRv instances)
                  to test:

                  master1.testReplication(suffix, replica, replica, replica, ...)

https://fedorahosted.org/389/ticket/48023

Reviewed by: nhosoi(Thanks!)

- - - - -
24cb7a8e by Mark Reynolds at 2017-10-11T15:24:09Z
Ticket 48022 - lib389 - Add all the server tasks

Description:  Added all the offline "tasks", and added all the slapi tasks
              to the module.

https://fedorahosted.org/389/ticket/48022

Reviewed by: nhosoi(Thanks!)

- - - - -
089d5493 by Mark Reynolds at 2017-10-11T15:24:09Z
Ticket 48028 - lib389 - add valgrind functions

Description: Added valgrind functions (enable, test, disable).  When
             we "enable" we copy in a valgrind ns-slapd wrapper,
             and backup the original ns-slapd.

             The "test" function just searches the output file for a
             string or regex.

             The "disable" function restores the original ns-slapd

https://fedorahosted.org/389/ticket/48028

Reviewed by: nhosoi(Thanks!)

- - - - -
eb13f511 by Mark Reynolds at 2017-10-11T15:24:09Z
Ticket 48028 - add valgrind wrapper for ns-slapd

Description:  Add a default valgrind wrapper for ns-slapd

https://fedorahosted.org/389/ticket/48028

Reviewed by: nhosoi(Thanks!)

- - - - -
e3ab42f9 by Mark Reynolds at 2017-10-11T15:24:09Z
Ticket 48029 - Add missing replication related functions

Description:  There was either missing or not fully implemented functions
              for disabling replication, deleting agreements, and deleting
              the changelog.

              Also added some helper functions to "grep" log files, or any file.

https://fedorahosted.org/389/ticket/48029

Reviewed by: tbordaz(Thanks!)

- - - - -
1fab1684 by Mark Reynolds at 2017-10-11T15:24:09Z
Fix string formating error

- - - - -
88134974 by Mark Reynolds at 2017-10-11T15:24:09Z
Add check for the new "stress" directory for "getDir()", and added a new function to open a new connection to the server

- - - - -
2bca12a6 by Mark Reynolds at 2017-10-11T15:24:09Z
Ticket 48238 - Add objectclass and attribute type query mechanisms

From: William Brown <wililam at adelaide.edu.au>
Date: Mon, 3 Aug 2015 11:40:41 +0930

https://fedorahosted.org/389/ticket/48238

Reviewed by: mreynolds

- - - - -
b9c441c0 by Mark Reynolds at 2017-10-11T15:24:09Z
Ticket 48236 - Add get effective rights helper to lib389

From: William Brown <wililam at adelaide.edu.au>
Date: Sat, 1 Aug 2015 13:12:10 +0930

Description: Add get effective rights search helper into dirsrv object

Reviewed by: mreynolds

https://fedorahosted.org/389/ticket/48236

- - - - -
2c9fcdd8 by Mark Reynolds at 2017-10-11T15:24:10Z
Add the package plugin dir to the DirvSrv object

- - - - -
9a086545 by Mark Reynolds at 2017-10-11T15:24:10Z
Ticket 48237 - Add lib389 helper to enable and disable logging services.

From: William Brown <wililam at adelaide.edu.au>
Date: Mon, 3 Aug 2015 10:05:06 +0930
Subject: [PATCH] Add log service enable and disable commands to
 lib389/brooker.py

https://fedorahosted.org/389/ticket/48237

Reviewed by: mreynolds

- - - - -
210573b3 by Mark Reynolds at 2017-10-11T15:24:10Z
Ticket 48239 - Fix for prefix allocation of un-initialised dirsrv objects

From: William Brown <william at adelaide.edu.au>
Date: Thu, 6 Aug 2015 10:11:25 +0930

Reviewed by: mreynolds

- - - - -
b83fddfa by Mark Reynolds at 2017-10-11T15:24:10Z
Add broooker function to set access log buffering on or off

- - - - -
a274ac82 by Mark Reynolds at 2017-10-11T15:24:10Z
Ticket 47848 - Add new function to create ldif files

Description:  Add function to create simple ldif files and set the
              owner and permissions as necessary

https://fedorahosted.org/389/ticket/48248

Reviewed by: rmeggins(Thanks!)

- - - - -
662031a3 by Mark Reynolds at 2017-10-11T15:24:10Z
From: William Brown <wililam at adelaide.edu.au>
Date: Thu, 6 Aug 2015 10:11:25 +0930
Subject: [PATCH] Example of commandline tools implementation for listing all
 attribute types and instances

Reviewed by: mreynolds

- - - - -
55555199 by Mark Reynolds at 2017-10-11T15:24:10Z
Ticket 48247 - Change the default user to 'dirsrv'

Description:  Change the default user to 'dirsrv' instead of using nobody.
              Then when we remove the last instance remove the user if
              lib389 added it.

              Also made some minor fixes.

https://fedorahosted.org/389/ticket/48247

Reviewed by: rmeggins(Thanks!)

- - - - -
4a3b5f30 by Noriko Hosoi at 2017-10-11T15:24:10Z
Ticket #48252 - (lib389) adding get_bin_dir and dbscan

Description: Adding APIs:
- get_bin_dir(sroot=None, prefix=None)
  returns <prefix>/usr/bin
- dbscan(self, bename=None, index=None, key=None)
  runs dbscan -f /path/to/db/bename/index{.db} [-k key | -K key]
  If index does not contain ".db", ".db" is added.
  If index is id2entry and key is digit, key is translated to "-K key".
  If index is not id2entry, key is translated to "-k key".

https://fedorahosted.org/389/ticket/48252

Reviewed by mreynolds at redhat.com (Thank you, Mark!!)

- - - - -
194e1db6 by William Brown at 2017-10-11T15:24:10Z
Reliability improvements to DS allocate functions

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
bf3ca65a by William Brown at 2017-10-11T15:24:10Z
Ticket 48259 - Add aci parsing utilities to lib389

Description:  Add aci parsing utilities, which will return an EntryAci.
              Can rebuild acis from dictionary data so that EntryAci
              objects can be edited and then saved back to ldap.
              Bindrules are not yet parsed, as this adds another layer
              of complexitiy. However the skeleton structures to parse
              these is in place.

https://fedorahosted.org/389/ticket/48259

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
84acf5dd by William Brown at 2017-10-11T15:24:11Z
Add dereference request control to lib389 for testing plugins.

- - - - -
70923e43 by William Brown at 2017-10-11T15:24:11Z
Ticket 48271 - Fix for self.prefix being none when SER_DEPLOYED_DIR is none https://fedorahosted.org/389/ticket/48271

Bug Description:  If SER_DEPLOYED_DIR is explicitly set to None,
        args.get(SER_DEPLOYED_DIR, self.prefix) is overwritten to None. This can
        cause tests in 389ds and lib389 to fail immediately.

Fix Descryption The .get method is only as a fall back if args doesn't contain
        the key, so this covers the case where SER_DEPLOYED_DIR is set
        incorrectly to None.

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
48d76ace by Mark Reynolds at 2017-10-11T15:24:11Z
Ticket 48273 - Improve valgrind functions

Description:  The previous valgrind functions were very limited in what you could do.
              Now, you can get the resutls file,m adn check it mulitple times, as
              well as pass in multiple pattern strings to check for in a single stack
              trace.

              Also added some helper constants for the memory leak or invalid access
              stacks.

https://fedorahosted.org/389/ticket/48273

Reviewed by: nhosoi(Thanks!)

- - - - -
03ef3aa3 by Viktor Ashirov at 2017-10-11T15:24:11Z
Update requirements.txt

pyasn1 and pyasn1-modules are required packages for lib389

- - - - -
258c6543 by Mark Reynolds at 2017-10-11T15:24:11Z
Ticket 48204 - update lib389 for python3

Description:  Updated lib389 for python3

https://fedorahosted.org/389/ticket/48204

Reviewed by: rmeggins(Thanks!)

- - - - -
8a74b849 by Mark Reynolds at 2017-10-11T15:24:11Z
Fix various errors in the changelog module

- - - - -
fcdf0937 by Viktor Ashirov at 2017-10-11T15:24:11Z
Ticket 48301 - add tox support

Description: Add tox.ini to automate testing with tox using
virtualenv and different versions of Python

https://fedorahosted.org/389/ticket/48301

Reviewed by: nhosoi (Thanks!)

- - - - -
b1934358 by Simon Pichugin at 2017-10-11T15:24:11Z
Ticket 48303 - Fix lib389 broken tests - aci_parse_test

Description: Fix the imports to the correct ones.
             Add Red Hat copyright block.
             Remove "Created on" block, because git contains
             this information.
             Refactore code to the pytest compatibility.

https://fedorahosted.org/389/ticket/48303

Review by: mreynolds (Thanks!)

- - - - -
973b59ac by Simon Pichugin at 2017-10-11T15:24:11Z
Ticket 48303 - Fix lib389 broken tests - agreement_test

Description: Fix the imports to the correct ones.
             Add Red Hat copyright block.
             Remove "Created on" block, because git contains
             this information.
             Remove hard coded variables, that reference to
             the local user home directory.
             Add missing docstrings for every test case.
             Remove redundant code from schedule test case.
             Fix expected exception assertions within create
             and schedule test cases.
             Add assert statement to status test case.
             Refactore code to the pytest compatibility.

https://fedorahosted.org/389/ticket/48303

Review by: mreynolds (Thanks!)

- - - - -
baefaadb by Simon Pichugin at 2017-10-11T15:24:11Z
Ticket 48309 - Fix lib389 lib imports

Description: Remove redundant imports, such as
             "from lib389.properties import SER_PORT" when
             "from lib389.properties import *" is presented.
             Group imports by standard library, related third party,
             lib389 library.

https://fedorahosted.org/389/ticket/48309

Reviewed by: mreynolds (Thanks!)

- - - - -
4c6319c3 by Simon Pichugin at 2017-10-11T15:24:11Z
Ticket 48303 - Fix lib389 broken tests - backend_test

Description: Fix the imports to the correct ones.
             Add Red Hat copyright block.
             Remove "Created on" block, because git contains
             this information.
             Add a logging.
             Add docstrings to the all tests.
             Divide the delete test case into two: for valid
             and invalid cases.
             Fix expected exception assertions within the create
             and the delete_invalid test cases.
             Add assert statement to the toSuffix test case.
             Refactore code to the pytest compatibility.

https://fedorahosted.org/389/ticket/48303

Review by: mreynolds (Thanks!)

- - - - -
0521b02b by William Brown at 2017-10-11T15:24:12Z
Ticket 48308 - Add __eq__ and __ne__ to Entry to allow fast comparison
https://fedorahosted.org/389/ticket/48308

Bug Description:
lib389's searches yield Entry objects. It's handy to be able to do:

pre = ds.search_s(somedn)
post = ds.search_s(somedn)

if pre == post:

    # evaluate in some way.

This will make some aspects of writing tests for dirsrv easier, especially when we are looking for changes in objects due to plugin effects.

Review by: nhosoi (Thanks!)

- - - - -
5319f15b by Simon Pichugin at 2017-10-11T15:24:12Z
Ticket 48318 - Do not delete a changelog while disabling a replication by suffix

Bug Description: If we have a few replicated suffixes on the single
                 instance, they will use the one changelog
                 ("cn=changelog5,%s" % DN_CONFIG).

                 If we disable the replication on the one of them by:
                 instance.replica.disableReplication(suffix=SUFFIX_2)

                 Then, according to the current logic, the changelog
                 will be deleted. But it is still required by another
                 replicated suffixes.

Fix Description: Remove the 'Delete the changelog' block from
                 disableReplication method of replica.py module.

https://fedorahosted.org/389/ticket/48318

Review by: mreynolds (Thanks!)

- - - - -
bbf5a4b4 by Mark Reynolds at 2017-10-11T15:24:12Z
Various fixes for REST API support

Add support for JSON entry representation.
Improve task class to store entry and dn of the task entry
Add post read controls when adding new index.

Reviewed by: wibrown(Thanks!)

- - - - -
326a7ed4 by Simon Pichugin at 2017-10-11T15:24:12Z
Ticket 48319 Fix ldap.LDAPError exception processing

Bug Description: Many lib389 modules contain the code block
                 like this:
                     except ldap.LDAPError as e:
                         self.log.fatal('Failed. Error: %s' %
                             e.message('desc'))
                         raise ldap.LDAPError

                 It causes problems for the debugging.

                 For example:
                 - ldap.LDAPError is a string object,
                 not a method or a function;
                 "e.message('desc'))" will cause
                 "TypeError: 'str' object is not callable".

                 - 'raise ldap.LDAPError' will raise
                 a new blank ldap.LDAPError object.

Fix Description: Replace e.message('desc') with str(e)
                 Replace 'raise ldap.LDAPError' with 'raise'

https://fedorahosted.org/389/ticket/48319

Review by: mreynolds (Thanks!)

- - - - -
8d539767 by Mark Reynolds at 2017-10-11T15:24:12Z
Ticket 48322 - Allow reindex function to reindex all attributes

Bug Description:  Currently the task reindex function requires an attribute
                  name.  This does now allow the task to reindex all of a
                  backend's indexes(like db2index.pl can do)

Fix Description:  If the attrname is not provided, search for all the backend's
                  attribute indexes, and reindex them.

https://fedorahosted.org/389/ticket/48322

Reviewed by: spichugi(Thanks!)

- - - - -
64b5a5b9 by William Brown at 2017-10-11T15:24:12Z
Ticket 48321 - Improve is_a_dn check to prevent mistakes with lib389 auth
https://fedorahosted.org/389/ticket/48321

Bug Description:   is_a_dn previously had a weak check to determine if a binddn
was in fact, a dn. This improves the check's quality to be far more thorough,
which is needed for the rest389 api.

Fix Description:  We now use the ldap.dn function to check the dn is valid, as
well as potentially allowing anonymous ("") as a valid dn for binding.

Author: wibrown

Review by: ???

- - - - -
34b4aec2 by William Brown at 2017-10-11T15:24:12Z
Ticket 48324 -  fix boolean capitalisation (one line)
https://fedorahosted.org/389/ticket/48324

Bug Description:   Fix false to False in utils.py

Author: wibrown

Review by: one liner, review not needed

- - - - -
de2ea202 by Mark Reynolds at 2017-10-11T15:24:12Z
Fixed pep8 errors in __init__.py, backend.py and mappingtee.py

- - - - -
a528d519 by Mark Reynolds at 2017-10-11T15:24:12Z
Ticket 48329 - add matching rule functions to schema module

Description:  Needed a way to access the matching rules in lib389

              Also did much needed pep8 cleanup

https://fedorahosted.org/389/ticket/48329

Reviewed by: wibrown(Thanks!)

- - - - -
2c1739ad by Simon Pichugin at 2017-10-11T15:24:12Z
Ticket 48303 - Fix lib389 broken tests

List of fixes through all the test suites:
- Remove SER_DEPLOYED_DIR assignment, because now it is
in __init__.py by default.
- Remove "Created on" block, because git contains this information.
- Add Red Hat copyright block.
- Add more logging.
- Add more docstrings.
- Fix the imports to the correct ones.
- Fix expected exception assertions using pytest.raises.
- Refactore code to the pytest compatibility.

tests/dirsrv_test.py:
- Refactor nearly whole test suite logic, because previous one
didn't have the proper test coverage.

tests/mappingTree_test.py:
- Remove _add_user and _mod_user functions,
because they are used nowhere.

tests/replica_test.py:
- Remove the test_delete test case, because replica.delete() is
obsolete, use replica.enableReplication() instead.
- Remove big block of commented code at the bottom.
- Add tests to the test_disableReplication test case,
now it is implimented.

tests/utils_test.py:
- Fix dictionary keys to the right ones, depends on what keys are
expected in the source code. For example: 'newhost' to 'hostname'.

https://fedorahosted.org/389/ticket/48303

Review by: mreynolds (Thanks!)

- - - - -
500c3a02 by Mark Reynolds at 2017-10-11T15:24:12Z
Ticket 48329 - Fix case-senstive scyheam comparisions

Description:  Fixed schema comparisions to be case-insensitive

https://fedorahosted.org/389/ticket/48329

- - - - -
21b96f1a by Mark Reynolds at 2017-10-11T15:24:12Z
Ticket 48335 - Add SASL support to lib389

Description:  Added SASL(GSSAPI) support to lib389, also added support
              for doing TLS.  Via William Brown, relaxed strict
              localhost lookup for GSSAPI

              Fixed pep8 errors in replica.py

https://fedorahosted.org/389/ticket/48335

Reviewed by: nhosoi(Thanks!)

- - - - -
89ae28bc by William Brown at 2017-10-11T15:24:12Z
Subject: [PATCH] Ticket 47840 - fix lib389 to use sbin scripts
 https://fedorahosted.org/389/ticket/47840

Bug Description:  Now that https://fedorahosted.org/389/ticket/528 is fixed,
the next step is to allow building the server with the instance specific scripts
disabled.

Fix Description:  As we do not install instance scripts by default we cannot
rely on their existance. This patch fixes the serverCmd function in lib389 to
use the sbin scripts, as well as providing two clitools for start/stop that can
be used to prove this functionality works.

Example:
python lib389/clitools/ds_start.py
python lib389/clitools/ds_stop.py

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
cdf22b0c by William Brown at 2017-10-11T15:24:13Z
Ticket 48343 - lib389 krb5 realm management
https://fedorahosted.org/389/ticket/48343

Bug Description:   We need to be able to test gssapi and other functions for 389.
Historically we used external krb5 services, but that led to potential issues
with keytab state and reliability.

This will allow us to create and destroy basic krb5 realms for testing purposes.

Fix Description:  We can use this in the following way:

from lib389.mit_krb5 import MitKrb5
krb = MitKrb5(realm="EXAMPLE.COM")
krb.create_realm()
krb.create_principal(principal="ldap/localhost.localdomain")
krb.create_keytab(principal="ldap/localhost.localdomain",
        keytab="/etc/dirsrv/slapd/ldap.keytab")
krb.destroy_realm()

While creating a DirSrv object, provided you have the REALM correctly configured
we automatically extract the keytab for the instance.

Author: wibrown

Review by: spichugi (Thanks very much!)

- - - - -
555744f7 by Mark Reynolds at 2017-10-11T15:24:13Z
Ticket 47840 - Fix regression

Description:  The previous commit switched to using the start/stop scripts
              from /sbin, instead of the instance scripts.  However, when
              using the sbin scripts you should specify the instance:

                 /sbin/start-dirsrv localhost

              This is automatically done in the instance scripts, and this
              behavior needs to be followed when using the sbin scripts.

              Note - this only seems to break when there are more than one
              instance on the system

https://fedorahosted.org/389/ticket/47840

Reviewed by: wibrown(Thanks!)

- - - - -
6391cf62 by Mark Reynolds at 2017-10-11T15:24:13Z
Ticket 48353 - Add Replication REST support to lib389

Description:  Needed to complete some existing functions, and fix
              several bugs in existing code.  Updated the properties
              using the REST representation values, added replication
              promotion/demotion functionality, replication agreement
              status functions, and did some code cleanup(pep8)

https://fedorahosted.org/389/ticket/48353

Reviewed by: wibrown & spichugi (Thanks!!)

- - - - -
8c92fb30 by William Brown at 2017-10-11T15:24:13Z
Ticket 48340 - Add basic monitor support to lib389 https://fedorahosted.org/389/ticket/48340

Ticket Description: Add monitoring utilites and helpers to lib389 to allow
us to check server state. This is a basic framework, with the intent of adding
more detailed inspection support later.

Author: wibrown

Review by: nhosoi (thanks!)

- - - - -
d021f3c6 by Mark Reynolds at 2017-10-11T15:24:13Z
Ticket 48246 - Adding license/copyright to lib389 files

Description:  Also fixed all the pep8 errors

https://fedorahosted.org/389/ticket/48246

Reviewed by: wibrown & spichugi(Thanks!!)

- - - - -
966f8048 by William Brown at 2017-10-11T15:24:13Z
Ticket 48361 - Expand 389ds monitoring capabilities

Bug Description:  Expand the capabilities of the 389ds monitoring tools in lib389

Fix Description: The monitor.py file was previously a placeholder with basic
functions. As we are looking to expand rest389, this needs to be expanded in
steps as we build the administrative interface.

https://fedorahosted.org/389/ticket/48361

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
9ffcb159 by Simon Pichugin at 2017-10-11T15:24:13Z
Ticket 48360 - Refactor the delete agreement function

Description: The delete agreement function of agreement.py module
doesn't cover all usage varieties and should be rewritten.

Now it requires suffix and replica(DirSrv object of the server
that the agreement points to). By some reasons, this replica object
may be missing, then we couldn't delete the agreement.

We don't have test coverage for that function.
It should be added to tests/agreement_test.py.

Also test_changes fails, fix is required.

Fix description: Change arguments of the delete agreement function to:
- suffix
- consumer_host - of the server that the agreement points to
- consumer_port - of the server that the agreement points to
or
- agmtdn - DN of the replica agreement

With that, we can clearly define the required agreement.

Add the delete test to tests/agreement_test.py

Also fix the changes test by adding time.sleep(2) before
change number checking, because there was not enough time
for updating change number on master.

https://fedorahosted.org/389/ticket/48360

Reviewed by: wibrown (Thanks!)

- - - - -
e11c9638 by Simon Pichugin at 2017-10-11T15:24:13Z
Ticket 48364 - Fix test failures

Description: New failures appear in the test suites. They should be
fixed as soon as possible. More new features are coming to lib389(like
REST or krb5 support) and they require a proper CI testing.

Fix description: Remove "+" sign from multiline string in the
lib389/backend.py, because it makes an impossible to replace string with
variables on the second line.

- tests/backend_test.py
Fix exception naming in the tests, because it was updated with recent
patches within lib389/backend.py module.

- tests/suffix_test.py
Refactor code to the pytest compatibility.

https://fedorahosted.org/389/ticket/48364

Reviewed by: wibrown (Thanks!)

- - - - -
260adce0 by Mark Reynolds at 2017-10-11T15:24:13Z
Ticket 48358 - Prepare lib389 for Fedora Packaging

Description:  Created spec file, and updated setup.py/setup.cfg.  Also
              restructured the code layout to all be under /lib389

https://fedorahosted.org/389/ticket/48358

Reviewed by: spichugi(Thanks!)

- - - - -
a3cae493 by Mark Reynolds at 2017-10-11T15:24:13Z
beta

- - - - -
95f7e410 by Mark Reynolds at 2017-10-11T15:24:13Z
Ticket 48358 - Make Fedora packaging changes to the spec file

Description:  Add the final changes to the spec file for specifying the url
              for the source, and License documentation

https://fedorahosted.org/389/ticket/48358

- - - - -
09f23e38 by Mark Reynolds at 2017-10-11T15:24:13Z
Ticket 58358 - Update spec file with pre-release versioning

Description:  Update the spec file for Fedora Packaging guidelines

- - - - -
83068624 by William Brown at 2017-10-11T15:24:13Z
Ticket 48371 - weaker host check on localhost.localdomain

Bug Description:  The utility of the localhost.localdomain check is diminshed
given that we can now setup instances with weaker host checks. Additionally,
the hostname checks really only matter for GSSAPI instances.

Fix Description:  This sets the default host to localhost, and weakens the
default check. Additionally, we add a new args parameter that enforces strong
hostname checks on instances.

https://fedorahosted.org/389/ticket/48371

Author: wibrown

Review by: spichugi (Thanks heaps!)

- - - - -
fe614b6a by Mark Reynolds at 2017-10-11T15:24:14Z
Ticket 48358 - Add new spec file

Description:  Add the new properly named python-lib389.spec file

https://fedorahosted.org/389/ticket/48358

- - - - -
be64c8dc by Mark Reynolds at 2017-10-11T15:24:14Z
Remove deprecated spec file

- - - - -
52f1b1c1 by William Brown at 2017-10-11T15:24:14Z
Ticket 48390 - RFE Improvements to lib389 monitor features for rest389

Bug Description:  We need to improve the monitor features of lib389, so that we
can expose more data for the use of the new admin system and cli tools. This
change will allow this to occur. Additionally, we can now base other advisory
tools on this for tuning etc.

Fix Description:  Implement more monitoring features.

https://fedorahosted.org/389/ticket/48390

Author: wibrown

Acknowledgements: I would like to thank the following staff of the University
of Adelaide for their advice on monitoring and needs that customers have with
ds systems:

* Mark Larsen
* David Monro
* Tim Bowen
* Matt Salkeld

Review by: ???

- - - - -
b1d4ba34 by Mark Reynolds at 2017-10-11T15:24:14Z
Update monitor module to catch exceptions around searches,
and to include a DN in the returned representation

- - - - -
317bdd0f by William Brown at 2017-10-11T15:24:14Z
Ticket 48401 - lib389 Entry hasAttr returs dict instead of false

Bug Description:   If the search is highly targeted, and self.data in Entry is
{}, hasAttr returns {} instead of False.

Fix Description:  The error is in the one line return function. We alter this
to be more paranoid to ensure that we are returning the correct result.

https://fedorahosted.org/389/ticket/48401

Author: wibrown

Review by: mreynolds (Thank you!)

- - - - -
349c8a66 by William Brown at 2017-10-11T15:24:14Z
Ticket 48401 - Revert typecheck

Bug Description:  cidict doesn't inherit collections.Mapping, so we always fail
hasAttr.

Fix Description:  Revert the type check, but retain the fix for returning {}

https://fedorahosted.org/389/ticket/48401

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
1623ec3d by Viktor Ashirov at 2017-10-11T15:24:14Z
Add missing dependencies for python-lib389

Ticket 48405 - python-lib389 in rawhide is missing dependencies

Description:  Add missing dependencies to python-lib389 spec file

https://fedorahosted.org/389/ticket/48405

Review by: mreynolds (Thanks!)

- - - - -
74596d0a by William Brown at 2017-10-11T15:24:14Z
Ticket 48408 - RFE escaped default suffix for tests

Bug Description:   Tickets such as 548 [0] sometimes required the escaped
default suffix. We should add this as a constant.

Fix Description:  Add the DEFAULT_SUFFIX_ESCAPED constant.

https://fedorahosted.org/389/ticket/48408

Author: wibrown

Review by: One line policy

- - - - -
c4c03362 by William Brown at 2017-10-11T15:24:14Z
Ticket 48415

Bug Description:   We commonly use encoded or other forms of our default suffix.
We should make these variables in case we change them.

Fix Description:  Add default domain parameter

https://fedorahosted.org/389/ticket/48415

Author: wibrown

Review by: One line rule

- - - - -
06eb8be7 by Mark Reynolds at 2017-10-11T15:24:14Z
Ticket 48419 - getadminport() should not a be a static method

Bug Description:  getadminport() was defined at astatic mehtod, but this function
                  was not part of any class.  This causes issues with sphinx
                  documentation.

Fix Description:  Remove the staticmethod decorator

https://fedorahosted.org/389/ticket/48419

Reviwed by: mreynolds (one line commit rule)

- - - - -
bfea8509 by William Brown at 2017-10-11T15:24:14Z
Ticket 48560 - Make verbose handling consistent

Bug Description:  The verbose flag doesn't always turn up verbosity when expected
. This is because we mishandled it in a number of places, didn't pass it to
children etc.

Fix Description:  Clean up our verbose flag handling in __init__.py

https://fedorahosted.org/389/ticket/48560

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
9439cd6b by William Brown at 2017-10-11T15:24:14Z
Ticket 48357 - clitools should standarise their args

Bug Description:  Ldap tools tend to follow some conventions, especialy in
389. These are
* -D for bindDN
* -Z for instance name
* -n for backend name
We should make the lib389 tools conform to this.

Fix Description:  Update the lib389 clitools to use standardised args.

https://fedorahosted.org/389/ticket/48357

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
d5c51310 by Simon Pichugin at 2017-10-11T15:24:14Z
Ticket 48407 - Add test coverage module for lib389 repo

Bug Description: We need some module that will show us how much lines of
code is covered by tests.

Fix Description: We can use pytest-cov plugin for this.

To get it work we need:
1) install python-pytest-cov package from Fedora or EPEL repos
or
1) install pytest-cov from pip
2) add "--cov-config .coveragerc --cov=lib389" to py.test command to
execute tests with coverage plugin

We shouldn't measure coverage for lib389/tests directory while executing
tests.
Patch creates .coveragerc config file to omit this directory.

https://fedorahosted.org/389/ticket/48407

Review by: wibrown (Thanks!)

- - - - -
249947e6 by Simon Pichugin at 2017-10-11T15:24:14Z
Ticket 48661 - Agreement test suite fails at the test_changes case

Bug Description: At the test_changes case at the agreement test suite,
change to the master can't be replicated to consumer due some reasons.
Error is not always reproducible. Sometimes change can be replicated.

Fix Description: Put RA Schedule to "Always" as the TearDown action in
the end of the previous test case (test_setProperties).

https://fedorahosted.org/389/ticket/48661

Review by: wibrown (Thanks!)

- - - - -
cbc89020 by William Brown at 2017-10-11T15:24:15Z
Ticket 48399 - Add helper makefile to lib389 to build and install

Bug Description:  It's not completely clear how to make a tarball or rpm from
the lib389 repo.

Fix Description:  This commit adds a helper make file and updates the readme
to support a build, install, test, rpm and srpm command. This will make it
easier to automate testing, and lower the barrier to entry to new developers.

https://fedorahosted.org/389/ticket/48399

Author: wibrown

Review by: vashirov

- - - - -
e07e489a by William Brown at 2017-10-11T15:24:15Z
Ticket 48751 - Improve lib389 ldapi support

Bug Description:  Previously lib389 didn't support ldapi. This limited some
connection options to always require a username or password, when autobind
is an option in some cases.

Fix Description:  This improves the support to allow ldapi, simple binds over
ldapi, and also to allow autobinding if the situation allows.

https://fedorahosted.org/389/ticket/48751

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
b29236c5 by William Brown at 2017-10-11T15:24:15Z
Ticket 48750 - Clean up logging to improve command experience

Bug Description:  Previously the loging was a bit too verbose by default. It
made for a lot of noise in command usage.

Fix Description:  This makes more log items require .verbose, and adds the
verbose flag to the clitools package.

https://fedorahosted.org/389/ticket/48750

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
cc8ab6e3 by Viktor Ashirov at 2017-10-11T15:24:15Z
Ticket 48765 - Change default ports for standalone topology

Bug description:
Some ports defined in _constants.py are invalid, because they're bigger
than 65535.

Fix description:
Change default ports for standalone topology with a range of ports
38930-38969.

https://fedorahosted.org/389/ticket/48765

Author: vashirov

Reviewed by: mreynolds (Thanks!)

- - - - -
ef612bf7 by William Brown at 2017-10-11T15:24:15Z
Ticket 48764 - Fix mit krb password to be random.

Bug Description:  The krb kdc password was not random. This is insecure.

Fix Description:  This makes the password random, and fixes our tests.

https://fedorahosted.org/389/ticket/48764

Author: wibrown

Review by: spichugi and mreynolds (Thanks)

- - - - -
13f9e954 by Mark Reynolds at 2017-10-11T15:24:15Z
Ticket 48660 - Add function to convert binary values in an entry to base64

Description:  There is a need to convert binary values in entries to
              base64 so they can easily be used in JSON representations.

https://fedorahosted.org/389/ticket/48660

Reviewed by: wibrown(Thanks!!)

- - - - -
77faa35a by William Brown at 2017-10-11T15:24:15Z
Ticket 48794 - lib389 build requires are on a single line

Bug Description:  Lib389 buildrequires are on a single line.

Fix Description:  Buildrequires should be on multiple lines.

https://fedorahosted.org/389/ticket/48794

Author: wibrown

Review by: spichugi (Thanks!!!)

- - - - -
d6fdb922 by William Brown at 2017-10-11T15:24:15Z
Ticket 48399 - Helper makefile is missing mkdir dist

Bug Description:  The lib389 make srpm file is missing the make dist folder
step which causes the git-archive to fail

Fix Description:  add the mkdir step

https://fedorahosted.org/389/ticket/48399

Author: wibrown

Review by: One line rule

- - - - -
f762622d by William Brown at 2017-10-11T15:24:15Z
Ticket 48399 - Helper makefile is missing mkdir dist

Bug Description:  The lib389 make srpm file is missing the make dist folder
step which causes the git-archive to fail.

Fix Description:  add the mkdir step, with -p, so it doesn't break.

https://fedorahosted.org/389/ticket/48399

Author: wibrown

Review by: One line rule

- - - - -
a6c7241c by William Brown at 2017-10-11T15:24:15Z
Ticket 48791 - format args in server tools

Bug Description:  A crash was found during a test:
                    # sometimes the server fails to start - try again
>                   rc = os.system("%s %s" % (fullCmd))
E                   TypeError: not enough arguments for format string

Fix Description:  Fix the parameters.

https://fedorahosted.org/389/ticket/48791

Author: wibrown

Review by: nhosoi (Thanks)

- - - - -
3b636b36 by William Brown at 2017-10-11T15:24:15Z
Ticket 48433 - Aci linting tools

Bug Description:  Directory Server acis are very complex. We should provide
linting tools that can detect and provide guidance around potential aci errors
that are common or easily made.

Fix Description:  Provide an aci linting test framework, and some initial lint
tests. Merges the aci tests to one file.

https://fedorahosted.org/389/ticket/48433

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
79cd8fa0 by William Brown at 2017-10-11T15:24:15Z
Ticket 48798 - lib389 add ability to create nss ca and certificate

Bug Description:  To effectively test SSL and other related components we need
to be able to reliably create a CA and Certificates.

Fix Description:  Add a helper to build a CA and create certs.

https://fedorahosted.org/389/ticket/48798

Author: wibrown

Review by: nhosoi (Thanks)

- - - - -
f8a75c54 by William Brown at 2017-10-11T15:24:15Z
Ticket 48798 - EL6 compat for lib389 tests for DH params

Bug Description:  Certain ldap modules cannot be imported on EL6

Fix Description:  Make a python version check for the module import

https://fedorahosted.org/389/ticket/48798

Author: wibrown

Review by: One line fix (Well, close to it)

- - - - -
a9b96a08 by William Brown at 2017-10-11T15:24:15Z
Ticket 48434 - lib389 logging tools

Bug Description:  Many tests or error cases need to be able to check the log
of the system to determine if the issue has occured on not.

Fix Description:  This adds an initial construction of a log parser and
tools to interact with access and error logs so that we can use these effectively
in tests.

https://fedorahosted.org/389/ticket/48434

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
dab92653 by William Brown at 2017-10-11T15:24:16Z
Ticket 48763 - backup should run regardless of existing backups.

Bug Description:  BackupFS needs to have clearBackupFs run first else it will
not run. This shouldn't need to be the case.

Fix Description:  This removes the "existing backup" check.

https://fedorahosted.org/389/ticket/48763

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
3f8914aa by William Brown at 2017-10-11T15:24:16Z
Ticket 48830 - Convert lib389 to ip route tools

Bug Description:  lib389 is using the deprecated ifconfig for a check.

Fix Description:  replace this with the correct call to ip route tools instead

https://fedorahosted.org/389/ticket/48830

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
5da35d41 by Viktor Ashirov at 2017-10-11T15:24:16Z
Ticket 48771 - lib389 - get ns-slapd version

Bug description:
Some tests should be executed only with 389-ds-base
that implements required features/has fixes. This can be done with
py.test skipif fixture. But we don't have a way to get the version
of ns-slapd that is currently used.

Fix description:
Add functions get_ds_version() and ds_is_older().
get_ds_version() returns a string like "1.3.4.8 B2016.043.2254"
ds_is_older(version) returns boolean value if the current version is
older than provided string.

https://fedorahosted.org/389/ticket/48771

Author: vashirov

Reviewed by: wibrown, mreynolds (Thanks!)

- - - - -
8320561e by Mark Reynolds at 2017-10-11T15:24:16Z
Ticket 48796 - add function to remove logs

Description:  There are CI tests that need to search DS logs.  To avoid harmless error
              codes triggered by server instance creation we need a function to remove
              the logs so we can start with a clean slate once the actual test begins.

              We also need to modify the ServerCmd function which expected to see an
              errors log.  So if the log is not present we just create an empty log and
              move on.

              Also added a helper function to print a date/time stamp, and added
              docstrings to various functions(to start to prep for sphinx documentation).

https://fedorahosted.org/389/ticket/48796

Reviewed by: nhosoi & wibrown (Thanks!!)

- - - - -
201ab483 by William Brown at 2017-10-11T15:24:16Z
Ticket 48434 - lib389 logging tools

Bug Description:  It would be useful to be able to search all logs of the server
even rotated ones.

Fix Description:  This adds the _archive methods which allows the searching of
older, rotated logs.

https://fedorahosted.org/389/ticket/48434

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
db573cb4 by William Brown at 2017-10-11T15:24:16Z
Ticket 48431 - lib389 integrate ldclt

Bug Description:  ldclt is an object generation and load testing tool. It is
very useful for exposing issues in 389-ds when load is applied in different
ways.

Fix Description:  This adds initial integration of ldclt. This allows creation
of a set of users from template, and the test to bind to them rapidly. More
functions will be added as they are needed over time.

https://fedorahosted.org/389/ticket/48431

Author: wibrown

Review by: spichugi (Thank you!)

- - - - -
a1b7649b by William Brown at 2017-10-11T15:24:16Z
Ticket ##### - Fixes for python 3

Bug Description:  Python 3 changes many types and behaviours with pyldap.

Fix Description:  Add wrappers to certain types. Begin to start fixing
python 3 behaviours for lib389

https://fedorahosted.org/389/ticket/48820

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
3593eac1 by William Brown at 2017-10-11T15:24:16Z
Ticket 48820 - Clitool rename

Bug Description: Clean up the clitools, as we use some of them in this feature

https://fedorahosted.org/389/ticket/48820

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
db0a7050 by William Brown at 2017-10-11T15:24:16Z
Ticket 48820 - Proof of concept of orm style mapping of configs and objects

Fix Description:  This changes the backend type and config to rely on the new
_mapped_object types. These abstract and essentially create an orm, allowing
us to easily derive and share common operations to our config types.

backend.py with the Backends and Backend classes are an excellent example of this

https://fedorahosted.org/389/ticket/48820

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
e9917234 by William Brown at 2017-10-11T15:24:16Z
Ticket 48820 - Move Encryption and RSA to the new object types

Fix Description:  Make an Encyrption and RSA types on the new objects. To make
this work, rejig some of the create code to move creation to the single item.

https://fedorahosted.org/389/ticket/48820

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
0f3898f1 by William Brown at 2017-10-11T15:24:16Z
Ticket 48820 - Fix tests to ensure they work with the new object types

Fix Description:  Fix tests to make sure they work with the new object types.
Some of these were just previously broken, but are now fixed.

https://fedorahosted.org/389/ticket/48820

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
60200a46 by William Brown at 2017-10-11T15:24:16Z
Ticket 48857 - Remove python-krbV from lib389

Bug Description:  python-krbV is no longer supported and not compatible with
python3.

Fix Description:  Remove krbV module, and use gssapi functions instead

https://fedorahosted.org/389/ticket/48857

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
b9c6fb55 by William Brown at 2017-10-11T15:24:16Z
Ticket 48434 - Fix for negative tz offsets

Bug Description:  In my infinite experience I only tested a positive timezone
(AEST +10). When run in a location with a negative offset (IE -4), this would
cause the log parser to crash.

Fix Description:  Fix the regex to accept a positive, negative tz

https://fedorahosted.org/389/ticket/48434

Author: wibrown

Review by: mreynolds (Thanks)

- - - - -
9cf26213 by William Brown at 2017-10-11T15:24:17Z
Ticket 48820 - Begin to test compatability with py.test3, and the new orm

Bug Description:  We should convert our existing python tests to py.test-3 and
they should work with the new ldap mapping types.

Fix Description:

* move all the random data genteration to passwd.py.
* Fix some backend handling data
* Fix broken self.conn calls in config
* Add some ldap filter injection defence to the DSLdapObject types
* Convert the test -k test_list lib389/lib389/tests/backend_test.py to the new backend type
* many more fixes for python 3 compatability

https://fedorahosted.org/389/ticket/48820

Author: wibrown

Review by: spichigi, mreynolds (Thanks!)

- - - - -
00c21c44 by William Brown at 2017-10-11T15:24:17Z
Ticket 48853 - Prerelease installer

Bug Description:  setup-ds.pl is pretty gnarly and old. Lets start thinking of
revamping it. This will be a long-multi year project, but time is short.

This adds a base installer, assertions, config parser, extensibility options.

It also works to install a minimal DS instance.

https://fedorahosted.org/389/ticket/48853

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
d21a7304 by William Brown at 2017-10-11T15:24:17Z
Ticket 48878 - pep8 fixes and fix rpm to build

Bug Description:   As we are about to release, we should check and run
pep8 on our code. There are also some other rpm fixes needed for
distribution, so we should basically have a tidy up before we release
the next version.

Fix Description:

* Fix all tests with pep8
* Add makefile helper for pep8
* exclude the sbin files as we aren't ready for them yet.

https://fedorahosted.org/389/ticket/48878

Author: wibrown

Review by: spichugi, mbasti (Thanks!)

- - - - -
fdc28879 by William Brown at 2017-10-11T15:24:17Z
Ticket 48878 - pep8 fixes part 2

Bug Description:   As we are about to release, we should check and run
pep8 on our code. There are also some other rpm fixes needed for
distribution, so we should basically have a tidy up before we release
the next version.

Fix Description:

* Fix all other components for pep8
* Fix a missing varible in config

https://fedorahosted.org/389/ticket/48878

Author: wibrown

Review by: spichugi, mbasti (Thanks!)

- - - - -
83ef8d7c by William Brown at 2017-10-11T15:24:17Z
Ticket 48878 - better style for backend in backend_test.py

Bug Description:  Thanks to mbasti:

"""
This change is harder to read with PEP8
-    topology.standalone.backends.create(None,
-                                       properties={
-                                            BACKEND_NAME: NEW_BACKEND_1,
-                                            'suffix':NEW_SUFFIX_1,
-                                            })
+    topology.standalone.backends.create(None, properties={
+ BACKEND_NAME: NEW_BACKEND_1,
+ 'suffix': NEW_SUFFIX_1,
+                                                         })
      ents = topology.standalone.backends.list()
      for ent in ents:
          topology.standalone.log.info("List(%d): backend %s" %

I suggest to use
topology.standalone.backends.create(
     None, properties={
         BACKEND_NAME: NEW_BACKEND_1,
         'suffix': NEW_SUFFIX_1,
     })
"""

Fix Description:  Apply the recommendation

https://fedorahosted.org/389/ticket/48878

Author: wibrown

Review by: spichugi, mbasti (Thanks!)

- - - - -
483654bd by William Brown at 2017-10-11T15:24:17Z
Ticket 48884 - Bugfixes for mapped object and new connections

Bug Description:  This patch corrects a number of behaviours around ldap
connections, and the mapped object type behaviour in certain conditions.
(Rdn set through properties, rather than argument)

Fix Description:  Fix handling of ldapurl, start_tls, and mapped_objects
when the rdn is part of properties.

https://fedorahosted.org/389/ticket/48884

Author: wibrown

Review by: nhosoi (Thanks)

- - - - -
f3b63f00 by William Brown at 2017-10-11T15:24:17Z
Ticket 48885 - Fix spec file requires

Bug Description:  The spec file did not requires all the libraries we needed.

Fix Description:  Add the requires for:

* python-pyasn1
* python-pyasn1-modules
* python-dateutil

https://fedorahosted.org/389/ticket/48885

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
a2465840 by William Brown at 2017-10-11T15:24:17Z
Ticket 48886 - Fix NSS SSL library in lib389

Bug Description:  On some platforms python-nss is amazingly broken. We can't
trust it, or use it.

Fix Description:  Re-write all checking operations to parse the shell output
of certutil and friends.

https://fedorahosted.org/389/ticket/48886

Author: wibrown

Review by: spichugi (Thanks)

- - - - -
72a1cdae by William Brown at 2017-10-11T15:24:17Z
Ticket 48888 - Correction to create of dsldapobject

Bug Description:  We did not allow properly setting of DN for singletons in
create, or the many object case with the factor. This corrects that behaviour.

Fix Description:  If _dn is set, use it, else use the RDn + BaseDN to create
the dn of the object to be created

https://fedorahosted.org/389/ticket/48888

Author: wibrown

Review by: mreynolds (Thanks)

- - - - -
b35e8c77 by Mark Reynolds at 2017-10-11T15:24:17Z
Ticket 48860 - Add replication tools

Description:  Created new replciation classes that use the new get/set
              mapping object class.  Also added a variey of replication
              tools

https://fedorahosted.org/389/ticket/48860

Reviewed by: firstyear(Thanks!)

- - - - -
1c01e66d by William Brown at 2017-10-11T15:24:17Z
Ticket 48910 - Fixes for backend tests and lib389 reliability.

Bug Description:  During the update to lib389 a number of tests broke. Repeat
runs of the tests as a whole, if interrupted can cause failing of the next run.

This is because we re-use port 54321, bit with a different server id, so setup-ds
fails.

Fix Description:
* Fix the backend object type
* Change the serverid of all tests to be consistent.

https://fedorahosted.org/389/ticket/48910

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
f7f50ea7 by William Brown at 2017-10-11T15:24:17Z
Ticket 48911 - Improve plugin support based on new mapped objects

Bug Description:  Previously managing plugins via lib389 was a very manual
task. This provides a level of automation and abstraction to that process.
It lays a foundation for making tests more succinct, and improving our admin
tools.

Fix Description:  Add the plugin types, and improve mapped_objects to be
able to map childobject types based on attributes.

https://fedorahosted.org/389/ticket/48911

Author: wibrown

Review by: spichugi (thanks!)

- - - - -
7a7ed858 by William Brown at 2017-10-11T15:24:17Z
Ticket 48911 - Plugin improvements for lib389

Bug Description:  We broke some legacy compatability. It's also useful to be
able to check the plugin status.

Fix Description:  Add the needed helpers, and add a status helper

https://fedorahosted.org/389/ticket/48911

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
722b0851 by William Brown at 2017-10-11T15:24:17Z
Ticket 48917 - Attribute presence

Bug Description:  Need a way to check for attribute presence. Add default equal
for entry aci

Fix Description:
* Add the def present to dsldapobject
* add the .get(,True) for equal in format_term

https://fedorahosted.org/389/ticket/48917

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
734a2144 by Mark Reynolds at 2017-10-11T15:24:18Z
Ticket 48923 - serverCmd timeout not working as expected

Bug Description:  When trying to start a server, and the startup fails,
                  the start-dirsrv command will actually hang for quite
                  a while, and the current timeout implementation does
                  not apply to this scenario.

Fix Description:  Instead of calling "os.system()", use subprocess and
                  a timer that will kill the process if it timeouts.

https://fedorahosted.org/389/ticket/48923

Reviewed by: nhosoi(Thanks!)

- - - - -
447e2017 by Mark Reynolds at 2017-10-11T15:24:18Z
Ticket 48923 - Fix additional issue with serverCmd

Description:  Check return code of "cmd" before entering loop.

              also fixed issues with valgrind ns-slapd wrapper

https://fedorahosted.org/389/ticket/48923

Reviewed by: mreynolds (one line commit rule)

- - - - -
11b38499 by Mark Reynolds at 2017-10-11T15:24:18Z
Fix valgrind functions

- - - - -
269ff667 by Mark Reynolds at 2017-10-11T15:24:18Z
Fix format error in print statement

One line commit rule

- - - - -
c7daf437 by Mark Reynolds at 2017-10-11T15:24:18Z
Ticket 48937 - Cleanup valgrind wrapper script

Description:  The valgrind wrapper script has a lot of unused code
              that should be cleaned up.

https://fedorahosted.org/389/ticket/48937

Reviewed by: nhosoi(Thanks!)

- - - - -
378fc969 by Noriko Hosoi at 2017-10-11T15:24:18Z
Bug 1347760 - Information disclosure via repeated use of LDAP ADD operation, etc.

Description: Adding 2 error cases.
1. When a SASL mapping is broken, the bind fails with INVALID_CREDENTIALS.
2. When a user is removed from the Directory Server, the bind fails with
   INVALID_CREDENTIALS.

https://bugzilla.redhat.com/show_bug.cgi?id=1347760

Reviewed by wibrown at redhat.com (Thank you, William!)

- - - - -
377583a0 by Simon Pichugin at 2017-10-11T15:24:18Z
Ticket 48382 - Fix serverCmd to get sbin dir properly

Description: On RHEL 6 test scripts would fail to execute sbin commands
like stop-dirsrv, start-dirsrv etc., because the path to them is wrong.

Fix description: Get sbin dir properly using the existing function
from utils.py

https://fedorahosted.org/389/ticket/48382

Reviewed by: nhosoi (Thanks!!)

- - - - -
92b319a8 by Simon Pichugin at 2017-10-11T15:24:18Z
Ticket 48832 - Add DirSrvTools.getLocalhost() function

Description: Some tests require us to know the exact localhost value
that is on the first place after 127.0.0.1.
(some Directory Server attributes is sensible at this matter)

https://fedorahosted.org/389/ticket/48832

Reviewed by: mreynolds (Thanks!)

- - - - -
7cdfc36d by Mark Reynolds at 2017-10-11T15:24:18Z
Ticket 48946 - openConnection should not fully popluate DirSrv object

Description:  When opening a connection to a server, there is no need to populate
              the DirDrv object which issues several searches.

https://fedorahosted.org/389/ticket/48946

Reviewed by: spichugi (Thanks!)

- - - - -
d1885bac by Mark Reynolds at 2017-10-11T15:24:18Z
Bump version to 1.0.2

- - - - -
cdc5a82d by xaellia at 2017-10-11T15:24:18Z
Ticket 48949 - configparser fallback not python2 compatible

Bug Description:  The python 3 dictionary and configure parser allows a default
option to be provided. Python 2 does not. We should support this for both
versions.

Fix Description:  Add a ._get_config wrapper that correctly supports the right
default fallback mechanism.

https://fedorahosted.org/389/ticket/48949

Author: xaellia

Review by: wibrown

- - - - -
c0ae31dd by xaellia at 2017-10-11T15:24:18Z
Ticket 48949 - os.makedirs() exist_ok not python2 compatible, added try/except

Bug Description:  os.makedirs on python3 takes an option that allows ignoring
the error if the directory does not exist in python2. This should be made more
portable by the use of the exception handling mechanism.

Fix Description:  Wrap the call in a try except to handle osError.

https://fedorahosted.org/389/ticket/48949

Author: xaellia

Review by: wibrown

- - - - -
e96c0ac2 by xaellia at 2017-10-11T15:24:18Z
Ticket 48949 - change default file path generation - use os.path.join

Bug Description:  Our path manipulation was done based on strings. This resulted
in a number of key locations being listed as //..../thing//file. This is not
very clean or professional, and may cause some issues with other tools.

Fix Description:  swap to using the python os.path.join mechanism which is able
to handle this correctly.

https://fedorahosted.org/389/ticket/48949

Author: xaellia

Review by: wibrown

- - - - -
4480913f by xaellia at 2017-10-11T15:24:18Z
Ticket 48949 - added copying slapd-collations.conf

Bug Description:  During the install, we missed copying slapd-collations.conf to
the target install.

Fix Description:  Add slapd-collations to the copy list.

https://fedorahosted.org/389/ticket/48949

Author: xaellia

Review by: wibrown

- - - - -
6557c258 by William Brown at 2017-10-11T15:24:19Z
Ticket 48949 - Fix ups for style and correctness

Bug Description:  We were provided with a method that set the name to "set"
rather than get. The method shouldn't be in the object, but as a standalone helper.
Additionally, os.path.join can take "path/dir" , "next/path", and join. We don't
need to seperate them up to a whole array.

Fix Description:  Fix the method to be get not set. change paths to have /
seperator when possible.

https://fedorahosted.org/389/ticket/48949

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
49bab42a by Mark Reynolds at 2017-10-11T15:24:19Z
Ticket 48952 - Restart command needs a sleep

Description:  There needs to be a small sleep between the stop and start
              of the restart command

https://fedorahosted.org/389/ticket/48952

Reviewed by: mreynolds (one line commit rule)

- - - - -
48008ae8 by William Brown at 2017-10-11T15:24:19Z
Ticket 48951 - dsadm and dsconf base files

Fix Description:  To unite all commands within lib389! Add the base poc of dsadm
and dsconf. dsadm is able to list instances running on the server.

These are based on arg parse, and are extensible, able to be tested with pytest
and take advantage of all the features of lib389.

-- Move the SetupDs object out of tools to a seperate module in
the class hierarchy.

-- Complete rewrite of how the ini for the pre-rel installer works
this allows us to autogenerate examples, better manager merging options, and
track defaults. In the future we'll be able to base our versioned defaults
on this via subclassing.

-- So that we can assert the correct behaviour of the new installer
this allows us to unit test it, and assert the installation is correct as
we expect.

-- Add dsconf backend handling, and cli tests for the new
tools. This way we can make sure end to end of the process is tested, and that
the behaviour is correct as expected. This include option management and
failure scenarios we can't currently test in the cli tools.

-- Based on Mark's comments, add connOnly to the test and the cli.

https://fedorahosted.org/389/ticket/48951

Author: wibrown

Review by: nhosoi, mreynolds (Thanks!)

- - - - -
4ddf0915 by Mark Reynolds at 2017-10-11T15:24:19Z
Ticket 47957 - Update the replication "idle" status string

Description:  The replication update status messages have changed, and
              this needs to be reflected in lib389

https://fedorahosted.org/389/ticket/48957

Reviewed by: mreynolds (one line commit rule)

- - - - -
087b11f2 by William Brown at 2017-10-11T15:24:19Z
Ticket 48951 - dsadm dsconfig status and plugin

Fix Description:  Add support for dsadm and dsconf to be able to check the
server status. Additionally, we add plugin support, and move fixtures to a
shared fixture module.

For example:

> dsadm instance status
Directory Server instance name : localhost
INFO:dsadm:Instance is running

> dsconf -D 'cn=Directory Manager' -H ldap://localhost plugin list
Enter password for cn=Directory Manager on ldap://localhost :

INFO:dsconf.plugin_list:7-bit check
INFO:dsconf.plugin_list:Account Policy Plugin
INFO:dsconf.plugin_list:Account Usability Plugin
...
INFO:dsconf.plugin_list:SSHA384
INFO:dsconf.plugin_list:SSHA512

https://fedorahosted.org/389/ticket/48951

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
5d832dca by William Brown at 2017-10-11T15:24:19Z
Ticket 48984 - Add lib389 paths module

Bug Description:  lib389 needs a way to consume the paths discovered by
defaults.inf. This allows us to consume that in the new installer, and in
test cases provided by lib389

Fix Description:  Add the paths module that searches wellknown locations for
defaults.inf. If not found, we error. At the moment, this assumes all installs
will follow the defaults.inf, and in the future we will add support for this
to read from dse.ldif if an instance is specified.

https://fedorahosted.org/389/ticket/48984

Author: wibrown

Review by: spichugi, mreynolds (thanks)

- - - - -
e3d870ef by William Brown at 2017-10-11T15:24:19Z
Ticket 48991 - Fix lib389 spec for python2 and python3

Bug Description:  Python3 is soon to be the default in fedora. We should be
ready for this, and lay the foundation for rhel adoption of python3 also

Fix Description:  Fix the spec file to create python2-lib389 and python3-lib389
on rhel we only generate the python2 version. Additionally, we gate our new
cli tools to be python3 only in these builds.

https://fedorahosted.org/389/ticket/48991

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
4c2ca2f2 by William Brown at 2017-10-11T15:24:19Z
Ticket 49005 - Update lib389 to work in containers correctly.

Bug Description:  lib389 and DS does not correctly detect or operate in a
container environment yet.

Fix Description:  Fix detection of systemd presence via the paths module. Fix
plugins to be able to be created and enabled correctly during installation.
Fix permissions of files we create and install.

https://fedorahosted.org/389/ticket/49005

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
6b78e08b by William Brown at 2017-10-11T15:24:19Z
Ticket 49007 - lib389 fixes for paths to use online values

Bug Description:  The test basic relied on some values that were not in lib389

Fix Description:  Update the values in lib389 for what test basic expects: While
we are messing about, may as well just update all the values and simplify out
the __initPart2 function.

https://fedorahosted.org/389/ticket/49007

Author: wibrown

Review by: mreynolds (thanks!)

- - - - -
086c95b4 by William Brown at 2017-10-11T15:24:19Z
Ticket 49010 - Lib389 fails to start with systemctl changes

Bug Description:  systemctl changed their api to status which broke the
lib389 wrapper. Pyldap doesn't work with bytes mode on fedora 24 and python 2.
Asan would not allow the server to start correctly without the env options

Fix Description:  Change the systemctl interface to use is-active, check pyldap
for python 3, and add the asan passthrough options.

https://fedorahosted.org/389/ticket/49010

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
a12b4507 by Simon Pichugin at 2017-10-11T15:24:19Z
Fix runUpgrade tool issues

Description: We should not lost any exception info.
Change 'assert False' to 'raise'.

Use sbin_dir parameter from defaults.inf instead of prefix.
RHEL 6 has setup-ds.pl under /usr/sbin, and if we install
Directory Server under '/' prefix current code would fail.
Also, sbin_dir makes much more sence then prefix in this case.

https://fedorahosted.org/389/ticket/49018

Reviewed by: nhosoi (Thanks, Noriko!)

- - - - -
fa477849 by William Brown at 2017-10-11T15:24:19Z
Ticket 48961 - Fix lib389 minor issues shown by 48961 test

Bug Description:  There were a small number of issues in lib389. Mainly in
reporting of errors with replication and how we deleted values.

Fix Description:  Fix the detection of a None to modify_s so that we can delete
values. Fix error message for replication.

https://fedorahosted.org/389/ticket/48961

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
df8ac382 by Simon Pichugin at 2017-10-11T15:24:19Z
Ticket 49024 - Fix paths in tools module

Description: Now we have paths module, so we can adjusted obsolete
hardcoded DS paths and fix some test failures with it.

Also, fix small issues with:
- ds-admin.pl variables;
- unsafe and consice command execution;
- fix lib dir path to the path in defaults.inf file

https://fedorahosted.org/389/ticket/49024

Reviewd by: nhosoi, wbrown (Thanks!)

- - - - -
453d5cb1 by Simon Pichugin at 2017-10-11T15:24:20Z
Ticket 49024 - Fix db_dir paths

Description: Wrong usage of db_dir path causes an instance restart
failure when plugins that envolve that directory are enabled.
Set them properly through the code.

https://fedorahosted.org/389/ticket/49024

Reviewed by: nhosoi, wibrown (Thanks!)

- - - - -
6abf7efe by Simon Pichugin at 2017-10-11T15:24:20Z
Ticket 49024 - Fix paths to the dbdir parent

Description: The parameter inst_dir was installed into defaults.inf with
a wrong value. It had caused critical failures during lib389 deployment.
Correct paths to use the dbdir parent.

https://fedorahosted.org/389/ticket/49024

Reviewed by: wibrown (Thanks!)

- - - - -
11de7279 by William Brown at 2017-10-11T15:24:20Z
Ticket 49022 - Lib389, py3 installer cannot create entries in backend

Bug Description:  The issue was that when we created the backend we were not
making the associated mapping tree.

Fix Description:  This fixes addresses a few issues. First, it addes the new
mapping Tree mapped object type, and renames the old one to legacy.

We add the integration between the backend and the mapping tree for creation
and deletion of the mt with the event.

We add a linting event and command to detect invalid backend configurations.

https://fedorahosted.org/389/ticket/49022

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
2e9428d9 by William Brown at 2017-10-11T15:24:20Z
Ticket 48707 - Implement draft-wibrown-ldapssotoken-01

Bug Description:  For our new webserver, we need a way to authenticate to the
directory server, while leaving the python webapplication minimal and without
security risk. To achieve this, we will allow DS to implement all security
controls, and the python rest server just passes data back and forth.

Fix Description:  This update allows lib389 to test the ldapssotoken module.
It allows assertion of properties on the rootdse, it adds the plugin handlers
so that we can create and enable the plugin, we add sasl plain support for
testing.

https://tools.ietf.org/html/draft-wibrown-ldapssotoken-01
https://fedorahosted.org/389/ticket/48707

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
bb94fd72 by Simon Pichugin at 2017-10-11T15:24:20Z
Ticket 47747 - Add topology fixtures module

Description: Add three topology fixtures in lib389/topologies.py.

It contains:
- topology_st - with topology_st.standalone inst;
- topology_m2 and topology_m4 - with dicts
topology_mN.ms["masterX"]
topology_mN.ms["masterX_agmts"]["mX_mY"]

To use it in our test we must import the module from lib389
and use it as a usual fixture afterwards.
Also, with a new structure we can easily run through masters
and agreements in loops.

https://fedorahosted.org/389/ticket/47747

Reviewed by: tbordaz, wibrown (Thanks!)

- - - - -
d74110e8 by Simon Pichugin at 2017-10-11T15:24:20Z
Ticket 47840 - Add InstScriptsEnabled argument

Description: Add an argument available for an allocation
during the instance creation process.

https://fedorahosted.org/389/ticket/47840

Reviewed by: wibrown (Thanks!)

- - - - -
ef2424eb by Simon Pichugin at 2017-10-11T15:24:20Z
Ticket 47747 - Add more topology fixtures

Description: Add topology fixtures to topologies.py:
- topology_m1c1
- topology_m1h1c1
- topology_m3

https://fedorahosted.org/389/ticket/47747

Reviewed by: wibrown (Thanks!)

- - - - -
a828e36c by Viktor Ashirov at 2017-10-11T15:24:20Z
Ticket 49060 - Increase number of masters, hubs and consumers in topology

Bug Description:
Currently only 10 masters, hubs and consumers can be created for
testing using lib389. We should increase that number to at least
60 (number of supported replicas in RHEL7.3).

Fix Description:
Generate constants for instances on the fly, instead of hardcoding.

https://fedorahosted.org/389/ticket/49060

Reviewed by: spichugi (Thanks!)

- - - - -
f599eeda by Simon Pichugin at 2017-10-11T15:24:20Z
Ticket 49055 - Fix debugging mode issue

Bug description: It is impossible to turn debugging mode on without
modifying lib389 code.

Fix description: Change the source of DEBUGGING constant from hard coded
to the one from environment variable named DEBUGGING.

https://fedorahosted.org/389/ticket/49055

Reviewed by: wibrown (Thanks!)

- - - - -
b388580d by William Brown at 2017-10-11T15:24:20Z
Ticket 49083 - Support prefix for discovery of the defaults.inf file.

Bug Description:  Due to the change to paths.py which consumes defaults.inf, this
broke custom prefix builds and tests with lib389.

Fix Description:  Allow paths.py to detect the presence of PREFIX, and to search
 PREFIX/share/dirsrv/inf/defaults.inf for the locations of DS. Once this file
is grabbed, everything else will "just work".

https://fedorahosted.org/389/ticket/49083

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
d231d46a by William Brown at 2017-10-11T15:24:20Z
Ticket 48413 - Improvements to lib389 for rest

Bug Description:  This improves a number of our resources for making rest389
easier to develop.

Fix Description:  The key change is in mapped_object, allowing any call to a
_json() to automatically map to the prefix, then wrapping the results.

https://fedorahosted.org/389/ticket/48413

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
67f960f7 by William Brown at 2017-10-11T15:24:20Z
Ticket 49087 - lib389 resolve jenkins issues

Bug Description:  Our jenkins tests were failing. Resolve the issues.

Fix Description:  The db2ldif and ldif2db scripts were flakey in some prefix
settings, so we avoid them as they are going to be removed eventually anyway.
Fix localhost, as localhost.localdomain literally does not exist on platforms
and it causes so many errors and pain.

https://fedorahosted.org/389/ticket/49087

Author: wibrown

Review by: nhosoi (Thanks!)

- - - - -
101cce51 by Simon Pichugin at 2017-10-11T15:24:20Z
Ticket 47747 - Add topology_i2 and topology_i3

Description: Add two more topologies with two and three standalone
instances. It would be more logical to call it "instance1", etc.,
but we areforced to use "standalone1" terminology for the compatibility
and efficiency.

You can refer to the instances with topology_i2.ins["standalon1"] dict.

Also you can enable a replication using REPLICAID_STANDALONE_* constants.

https://fedorahosted.org/389/ticket/47747

Reviewed by: lkrispen (Thanks!)

- - - - -
51c7218b by Mark Reynolds at 2017-10-11T15:24:21Z
Fixed regression with offline db2ldif

During an offlien importhte server istopped.  Someone removed the start after the import was complete

- - - - -
0ca62a24 by William Brown at 2017-10-11T15:24:21Z
Ticket 49103 - python 2 support for installer

Bug Description:  The python installer tools were originally targeted at python
3 only. Due to the slow adoption of python 3, and it's various issues, we can
not solely rely on it's presence. As a result we need to support python 2
as well.

Fix Description:  Add support for python 2 in the new installer code. This
was mainly related to a number of assumptions present in the init code, and
the use of set manipulations in arrays that is not possible in python2

https://fedorahosted.org/389/ticket/49103

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
120b1073 by William Brown at 2017-10-11T15:24:21Z
Ticket 49101 - Python 2 generate example entries

Bug Description:  In order to move to the new python installer, we need feature
parity with the existing perl installer. Currently, we are missing one feature,
which is sample entry creation. We need this for our test cases to run.

However, we also don't want to be stuck on the current set of sample entries.
They suck pretty hard, and the acis are just plain terrible. Yet our tests and
customers may depend on them.

Fix Description:  This contains multiple parts to solve the problem.

First, we add a set of identity management objects for groups, ous, users,
services accounts and more. This will greatly simplify our test code, and what
we can create.

Next, we add a set of managers that can apply version configs or sample entries.

We then wire in the capability for instance.create() to use the python installer,
be it via explicit request or environment variable (PYINSTALL) being set.

This way, we can now use the python installer experimentally during tests. A number
of my tests show this already passes.

The great benefit of using python to create the sample entries and configs, is
that they are programatic, versioned, can be unit tested, refactored, they are
portable, no more text file manipulation. We can also use the same code for *upgrade*
as we use for installs, making our tools and proceedures more consistent and
reliable.

A key point of this is that we will be able to install a 1.3.7 server, but with
1.3.6 defaults, or a 1.4.x server with 1.3.7 settings etc. New installs can be
kept in sync with older replicas. Once they are upgraded, we can then do proper
upgrades with the same code that does installs.

This also paves the way for us to add basic user and object management tools for
those unwilling to go the whole way into an IPA install.

https://fedorahosted.org/389/ticket/49101

Author: wibrown

Review by: spichugi (Thanks)

- - - - -
0bef150e by Noriko Hosoi at 2017-10-11T15:24:21Z
dbscan - Support additional options (-t truncate -R)

Thanks to wibrown at redhat.com for his ack.

- - - - -
a9a9237c by William Brown at 2017-10-11T15:24:21Z
Ticket 49126 - DIT management tool

Bug Description:  This adds a tool allowing us to manage items in the DIT of
an LDAP server.

Fix Description:  This tool is able to manage users, groups, ous and posixgroups.
Additionally, the functionality for schema management was ported into the dsconf
tool.

https://fedorahosted.org/389/ticket/49126

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
54acf0dd by William Brown at 2017-10-11T15:24:21Z
Ticket 2 -  pytest mark with version relies on root

Bug Description:  pytest checks the ds version by trying to open
nsslapd.

Fix Description:  use defaults.inf.

https://pagure.io/lib389/issue/2

Author: wibrown

Review by: vashirov (Thanks!)

- - - - -
0a82814c by William Brown at 2017-10-11T15:24:21Z
Ticket 49137 - Add sasl plain tests, lib389 support

Bug Description:  We need to test that SASL plain works in DS.

Fix Description:  To do this, we need to fix sasl in lib389, we
correct openConnection to use more than just gssapi, we fix
start tls to work over multiple connections.

https://pagure.io/389-ds-base/issue/49137

Author: wibrown

Review by: mreynolds (THanks!)

- - - - -
4ef5692c by William Brown at 2017-10-11T15:24:21Z
Ticket 4 - Cert detection breaks some tests

Bug Description:  Certdetection attempts to open a connection and read
cn=config in cases when we aren't able to access the attributes.

It's not worth it :(

Fix Description:  Disable the detection.

https://pagure.io/lib389/issue/4

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
72098713 by William Brown at 2017-10-11T15:24:21Z
Ticket 5 - Fix container build on fedora

Bug Description:  We need lib389 to help us build containerised versions of DS.
To resolve this, we need to fix a number of fedora build issues with the rpm.
Additionally, containers don't allow docs install, so we have to be able to
generate a working example setup.inf from the cli tools.

Fix Description:  Remove shebangs that cause rpm to generate incorrect
dependencies, shutdown ds at end of a container install, fix the example
generator to uncomment needed options.

https://pagure.io/lib389/issue/5

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
cb65e935 by William Brown at 2017-10-11T15:24:21Z
Ticket 6 - Bump lib389 version 1.0.4

- - - - -
c53e4365 by William Brown at 2017-10-11T15:24:21Z
Ticket 49172 - Allow lib389 to read system schema and instance

Bug Description:  lib389 only read instance schema despite the update
to dual locations in 1.3.6.

Fix Description:  Glob both paths.

https://pagure.io/389-ds-base/issue/49172

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
708adf46 by Viktor Ashirov at 2017-10-11T15:24:21Z
Ticket 49172 - Allow lib389 to read system schema and instance

Fix Description:  Fix a typo

https://pagure.io/389-ds-base/issue/49172

Review by: wibrown (Thanks!)

- - - - -
38345dbf by Simon Pichugin at 2017-10-11T15:24:21Z
Issue 7 - Add pause and resume methods to topology fixtures

Description: For testing purposes, sometimes we need to pause the
replication on a big amount of agreements (4 and more, for instance).
Add the methods to the TopologyMain class of lib389/topologies.py module.

https://pagure.io/lib389/issue/7

Reviewed by: wibrown (Thanks!)

- - - - -
db745aa7 by William Brown at 2017-10-11T15:24:22Z
Ticket 9 - Convert readme to MD

Fix Description:  Convert readme to MD

https://pagure.io/lib389/issue/9

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
f3b34087 by William Brown at 2017-10-11T15:24:22Z
Ticket 10 - Improve command line tool arguments

Bug Description:  Previously our tools were a poc, and didn't
have very good layout of commands.

Fix Description:  Improve these, to be of the form:

<command> <instance> <action> [<sub action>] <options>

For example:

usage: dsadm [-h] [-v] instance {restart,start,stop,status,db2index} ...

positional arguments:
  instance              The name of the instance to act upon
  {restart,start,stop,status,db2index}
                        action
    restart             Restart an instance of Directory Server, if it is
                        running: else start it.
    start               Start an instance of Directory Server, if it is not
                        currently running
    stop                Stop an instance of Directory Server, if it is
                        currently running
    status              Check running status of an instance of Directory
                        Server
    db2index            Initialise a reindex of the server database. The
                        server must be stopped for this to proceed.

usage: dsidm instance user [-h] {list,get,get_dn,create,delete} ...

positional arguments:
  {list,get,get_dn,create,delete}
                        action
    list                list
    get                 get
    get_dn              get_dn
    create              create
    delete              deletes the object

usage: dsidm instance user create [-h] [--uid [UID]] [--cn [CN]] [--sn [SN]]
                                  [--uidNumber [UIDNUMBER]]
                                  [--gidNumber [GIDNUMBER]]
                                  [--homeDirectory [HOMEDIRECTORY]]

optional arguments:
  -h, --help            show this help message and exit
  --uid [UID]           Value of uid
  --cn [CN]             Value of cn
  --sn [SN]             Value of sn
  --uidNumber [UIDNUMBER]
                        Value of uidNumber
  --gidNumber [GIDNUMBER]
                        Value of gidNumber
  --homeDirectory [HOMEDIRECTORY]
                        Value of homeDirectory

https://pagure.io/lib389/issue/10

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
2444cde8 by William Brown at 2017-10-11T15:24:22Z
Ticket 15 - Improve instance configuration ability

Bug Description:  We should be able to create and configure an instance
from python, rather than template-dse.ldif. This gives us the ability
to *install* and *upgrade* to a specific base version of the DS server.

Fix Description:  Implement stubs for our plugins, and prove the operation
works by enabling whoami during a minimal python install.

https://pagure.io/lib389/issue/15

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
3939a3fc by William Brown at 2017-10-11T15:24:22Z
Ticket 13 - Add init function to create new domain entries

Bug Description:  When we create a new backend, we were missing a
way to populate the entries for usage.

Fix Description:  Add an initialise command to dsidm which is
capable of creating sample entries based on the version supplied
or defaulting to the highest available version.

https://pagure.io/lib389/issue/13

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
4e20a696 by William Brown at 2017-10-11T15:24:22Z
Ticket 14 - Remane dsadm to dsctl

Bug Description:  Rename dsadm to dsctl

https://pagure.io/lib389/issue/14

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
e285341c by Simon Pichugin at 2017-10-11T15:24:22Z
Issue 16 - Reset InstScriptsEnabled argument during the init

Bug description: the InstScriptsEnabled argument is reused for the next
instance setups and it affects script directory creation.

Fix description: Reset the InstScriptsEnabled argument before instance
setup.

https://pagure.io/lib389/issue/16

Reviewed by: wibrown (Thanks!)

- - - - -
1b856ed7 by William Brown at 2017-10-11T15:24:22Z
Ticket 14 - Remane dsadm to dsctl

Bug Description:  Rename dsadm to dsctl
        I forgot to provide the dsctl files. They are in this
        commit.

https://pagure.io/lib389/issue/14

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
8eb8e5c7 by William Brown at 2017-10-11T15:24:22Z
Ticket 19 - Missing file and improve make

Bug Description:  This fixes a missing file in the RPM specfile.
This also makes the local makefile better for docker containers.

Fix Description:  Fix the renamed README line in spec, and improve
the topdir in rpmbuild

https://pagure.io/lib389/issue/19

Author: wibrown

Review by: vashirov (Thanks!)

- - - - -
39fe2ae4 by Ilias Stamatis at 2017-10-11T15:24:22Z
Ticket 20 - Use the DN_DM constant instead of hard coding its value

Bug Description:  We have defined the DN_DM string constant but we hard code
its value in a few places instead of using it.

Fix Description:  Replace the hard coded occurrences with the DN_DM constant.

https://pagure.io/lib389/issue/20

Author: Ilias95

Review by: wibrown (Thanks for your patch!)

- - - - -
5f02b93d by William Brown at 2017-10-11T15:24:22Z
Ticket 19 - missing readme.md in python3

Bug Description:  rpm build breaks on fedora

Fix Description:  missing readme.md on python 3

https://pagure.io/lib389/issue/19

Author: wibrown

Review by: one line fix, identical to python 2 fix.

- - - - -
3bb3f656 by Simon Pichugin at 2017-10-11T15:24:22Z
Issue 22 - Specify a basedn parameter for IDM modules

Description: We need a way to create our IDM users and groups in a
custom suffix. Now it is hard coded to 'ou=people' for users,
'ou=groups' for groups, etc.

Also, it is important to do not break current functionality.
CLI IDM depends on it. The parameter should remain optional,
with a default set to 'ou=people' for users, 'ou=groups' for groups, etc.

Also, fix test module names, so they can be successfully
discovered by pytest.

https://pagure.io/lib389/issue/22

Reviewed by: wibrown (Thanks!)

- - - - -
6fd69ce8 by Ankit yadav at 2017-10-11T15:24:22Z
Ticket 1 - cn=config comparison

Bug Description: Once we have enough plugin and index code,
we should be able to use dsconf to compare the state of two
instances. This will let us check for differing plugin, index,
configuration values between two servers.

Fix Description: This patch has a working compare function and
tests for comparing user objects.

https://pagure.io/lib389/issue/1

Author: ankity10

Review by: wibrown (Thanks Ankity!)

- - - - -
adede2bb by Ilias Stamatis at 2017-10-11T15:24:22Z
Ticket 23 - Use DirSrv.exists() instead of manually checking for instance's existence

Description: Manual instance existence checks can be replaced by calls
to DirSrv.exists().

https://pagure.io/lib389/issue/23

Author: Ilias95

Review by: wibrown (Thanks for the patch!)

- - - - -
7403ef0d by Mark Reynolds at 2017-10-11T15:24:22Z
Issue 25 - Fix RUV __repr__ function

Description:  This function never worked and had the wrong parenthesis grouping

https://pagure.io/lib389/issue/25

Reviewed by: firstyear(Thanks!)

- - - - -
0d20731c by Ilias Stamatis at 2017-10-11T15:24:23Z
Issue 24 - Join paths using os.path.join instead of string concatenation

https://pagure.io/lib389/issue/24

Author: Ilias95

Review by: wibrown (Thanks Ilias!)

- - - - -
7f85b91f by William Brown at 2017-10-11T15:24:23Z
Ticket 26 - improve lib389 sasl support

Bug Description:  Our sasl support was lacking, and in some cases
didn't work due to entry issues.

Fix Description:  Improve the support for rootdse, reseting config
values, and querying objects.

https://pagure.io/lib389/issue/26

Author: wibrown

Review by: spichugi, mreynolds (Thanks!)

- - - - -
66f3e7fe by William Brown at 2017-10-11T15:24:23Z
Ticket 21 - Missing serverid in dirsrv_test due to incorrect allocation

Bug Description:  The dirsrv_test attempts to build and destroy instances,
but it's fixture didn't allocate the instance. Due to changes in lib389
this caused exceptions.

Fix Description:  Move the allocation to the fixture, and remove the init
to allocated state test.

https://pagure.io/lib389/issue/21

Author: wibrown

Review by: ilias95, mreynolds (Thanks!)

- - - - -
5a794beb by Ankit Yadav at 2017-10-11T15:24:23Z
Ticket 1 - cn=config comparison

Bug Description: Once we have enough plugin and index code,
we should be able to use dsconf to compare the state of two
instances. This will let us check for differing plugin, index,
configuration values between two servers.

Fix Description: This patch has tests for comparing user objects of
two different DS instances and master to master replicated DS instances.
This patch also includes tests for cn=config comparison of two different DS instances.

https://pagure.io/lib389/issue/1

Author: ankity10

Review by: wibrown (Thank you!)

- - - - -
ad3e726a by Simon Pichugin at 2017-10-11T15:24:23Z
Issue 27 - Add a module for working with dse.ldif file

Description: For some tests we need a way to parse and edit dse.ldif
file. For starters, it will be nice to have next operations support:
- get - Return attribute values under a given entry;
- add - Add an attribute under a given entry;
- delete - Delete singlevalued or multivalued attributes under a given entry;
- replace - Replace attribute values with a new one under a given entry.

Add tests to lib389/tests/dseldif_test.py

https://pagure.io/lib389/issue/27

Reviewed by: wibrown (Thanks!)

- - - - -
949e4b12 by William Brown at 2017-10-11T15:24:23Z
Ticket 1 - Fix missing dn / rdn on config.

Bug Description:  During the change on #26 we changed the way
that we assert some vallues from the DB. This has exposed
an issue in the way that we handle RDN on config

Fix Description:  Since we always know the RDN and DN of cn=config
we can just provide it.

https://pagure.io/lib389/issue/1

Author: wibrown

Review by: ankity10, ilias95 (verified functionality)

- - - - -
ad647873 by William Brown at 2017-10-11T15:24:23Z
Ticket 28 - Change default objectClasses for users and groups

Bug Description:  Previously we had to use inetUser on users
for memberOf, and no option existed for groups.

Fix Description:  With the addition of nsMemberOf we can support users
and groups with this objectClass. As well, it allows us to remove
inetUser because it's attributes are supplied by the other classes
on the object.

https://pagure.io/lib389/issue/28

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
ae8f4277 by William Brown at 2017-10-11T15:24:23Z
Ticket 29 - fix incorrect format in tools

Bug Description:  Can not concat str and int

Fix Description:  change error to use string formatting

https://pagure.io/lib389/issue/29

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
d7ea68d8 by William Brown at 2017-10-11T15:24:23Z
Ticket 30 - Add initial support for account lock and unlock.

Bug Description:  Add support for account lock and unlock to
eventually replace ns-inactivate, ns-activate and ns-accountlock.

Fix Description:  Add a new account type that looks for any object
that supports userPassword. It can lock / unlock them. Additionally,
subclass UserAccount on this so inherit the account lock mech.

https://pagure.io/lib389/issue/30

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
63cdb504 by Simon Pichugin at 2017-10-11T15:24:23Z
Issue 27 - Improve dseldif API

Description: Return None instead of ValueError, if we haven't found
an attribute during a get operation.
During replace operation, if there is no attribute - just log an info.

https://pagure.io/lib389/issue/27

Reviewed by: wibrown (Thanks!)

- - - - -
044216f0 by William Brown at 2017-10-11T15:24:23Z
Ticket 28 - userAccount for older versions without nsmemberof

Bug Description:  Older DS versions don't support nsmemberof

Fix Description:  Make a version check on the objectClass

https://pagure.io/lib389/issue/28

Author: wibrown

Review by: spichugi, mreynolds (Thanks)

- - - - -
0b12f35b by Simon Pichugin at 2017-10-11T15:24:23Z
Issue 27 - Fix get function in tests

Description: Function 'dseldif.get' now returns None,
if we haven't found an attribute. Fix tests accordingly.

https://pagure.io/lib389/issue/27

Reviewed by: wibrown (Thanks!)

- - - - -
b76008ef by William Brown at 2017-10-11T15:24:23Z
Ticket 32 - Add TLS external bind support for testing

Bug Description:  Previously it was difficult to test TLS external
binds. We should add this to lib389 before any refactor of the
certmap code.

Fix Description:  This covers a number of locations. Importantly:

* The ability to configure certmaps programatically via config.
* The ability to generate and extract user certificates signed
    by the internal CA from lib389
* The ability to perform an external TLS bind.

https://pagure.io/lib389/issue/32

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
608a8b6e by William Brown at 2017-10-11T15:24:23Z
Ticket 60 - add dsrc to dsconf and dsidm

Bug Description:  Previously we had to always specifiy a backend to
the dsidm commands. As well, many options like TLS auth were not accesible.

Fix Description:  adding a ~/.dsrc allows an admin not only to
shorthand connect to instances IE:

dsidm -v -b dc=example,dc=com -D 'cn=Directory Manager' ldap://localhost ....

becomes:

dsidm -v localhost ....

It allows us to specify *multiple instances* (ie can swap identities easily)
and allows us to use SASL options like TLS external from our CLI tools.

https://pagure.io/lib389/issue/60

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
c0e4dbeb by William Brown at 2017-10-11T15:24:24Z
Ticket 62 - dirsrv offline log

Bug Description:  dirsrv log would use the online ldap server to lookup
the access and error path, but this does not work when instance is
offline.

Fix Description:  Convert this to use the ds_paths module that does
correctly account for this. Improve offline state handling so that
we can check this properly.

https://pagure.io/lib389/issue/62

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
acae4c16 by William Brown at 2017-10-11T15:24:24Z
Ticket 63 - lib389 python 3 fix

Bug Description:  In def open for python 3 we introduced a bynes
vs str issue in tls setopt

Fix Description:  wrap the offending code in ensure str

https://pagure.io/lib389/issue/63

Author: wibrown

Review by: ???

- - - - -
2129030f by William Brown at 2017-10-11T15:24:24Z
Ticket 63 - part 2, agreement test

Bug Description:  This resolves a number of issues in the python
3 replication agreement test.

Fix Description:  Fix bytes vs str when needed, and replace some
raw ldap calls with object types.

https://pagure.io/lib389/issue/63

Author: wibrown

Review by: ???

- - - - -
0a8808d9 by Sankar Ramalingam at 2017-10-11T15:24:24Z
Ticket 65 - Add m2c2 topology

Bug Description: Replication topology for two master and two consumers
is not available.

Fix Description: Add m2c2 replication topology.

https://pagure.io/lib389/issue/65

Author: Sankar Ramalingam

Reviewed by: spichugi

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
8e4f0baa by William Brown at 2017-10-11T15:24:24Z
Ticket 50 - Add db2* tasks to dsctl

Bug Description:  To make dsctl complete, we need to add the
various db2* tasks to allow backup and restore.

Fix Description:  Add the tasks, along with their tests and
some minor fixes to lib389 to support there.

https://pagure.io/lib389/issue/50

Author: wibrown

Review by: tbordaz, ilias95 (Thanks!)

- - - - -
52cd39a7 by Ilias Stamatis at 2017-10-11T15:24:24Z
Issue 31 - Initial MemberOf plugin support

Description: Add initial support for MemberOf by implementing
the basic methods for configuring the plugin and writing dsconf
hooks to manage it from the cli.

https://pagure.io/lib389/issue/31

Author: Ilias95

Review by: wibrown (Thank you very much!)

- - - - -
ef6e6093 by William Brown at 2017-10-11T15:24:24Z
Ticket 69 - add specfile requires

Bug Description:  lib389 requires iproute

Fix Description:  add this to the requires

https://pagure.io/lib389/issue/69

Author: wibrown

Review by: ilias95, mreynolds (Thanks!)

- - - - -
ad35586f by William Brown at 2017-10-11T15:24:24Z
Ticket 66 - expand healthcheck for Directory Server

Bug Description:  In order to aid admins, we should be able to offer
advice about the status of their servers.

Fix Description:  This expands the healthcheck command to include
encryption and password options. It changes the command name (from
lint to healthcheck), and add's generic wrappers to run these
more easily.

https://pagure.io/lib389/issue/66

Author: wibrown

Review by: ilias95, spichugi (Thanks!)

- - - - -
f41a430b by Ilias Stamatis at 2017-10-11T15:24:24Z
Issue 31 - Add functional tests for MemberOf plugin

Description: Add tests for testing the functionality of MemberOf
plugin. These tests make sure the plugin behaves properly based
on its configuration.

https://pagure.io/lib389/issue/31

Author: Ilias95

Review by: wibrown (Thanks, great work!)

- - - - -
9fe1a7ce by Ilias Stamatis at 2017-10-11T15:24:24Z
Issue 31 - Add status command and SkipNested support for MemberOf

Description:
- Add a generic function for displaying plugin's status.
- Make sure that existing Plugin.status() works properly with py3.
- Add support for configuring memberOfSkipNested attr.
- Add dsconf support and cli tests for memberOfSkipNested attr.
- Define must-have attributes for MemberOf.

https://pagure.io/lib389/issue/31

Author: Ilias95

Review by: wibrown (Thanks, great work!)

- - - - -
1ce515a5 by William Brown at 2017-10-11T15:24:24Z
Ticket 50 - typo in db2* in dsctl

Bug Description:  A typo was made that affected the addition
of the arguments to the cli.

Fix Description:  Fix the typos

https://pagure.io/lib389/issue/50

Author: wibrown

Review by: one line rule

- - - - -
5f1fbb83 by Mark Reynolds at 2017-10-11T15:24:24Z
Ticket 70 - Improve repl tools

Description:  Improve the repltools and a small change to the disrv_log for handling
              access logs that do not use the high resolution time.

              Thanks to lkispenz for adding improvements to the checkCSN function.

https://pagure.io/lib389/issue/70

Reviewed by: firstyear(Thanks!)

- - - - -
a9905306 by William Brown at 2017-10-11T15:24:24Z
Ticket 67 - get attr by type

Bug Description:  In python 3 the change to utf8 has caused us some
difficulty. As a result, to help ease this, we need to be able to retrieve
attributes by types in DS

Fix Description:  In mapped object, add the ability to request types
for searches when we perform them.

https://pagure.io/lib389/issue/67

Author: wibrown

Review by: ilias95, spichugi (Thanks!)

- - - - -
0cb4c667 by William Brown at 2017-10-11T15:24:24Z
Ticket 59 - lib389 support for index management.

Bug Description:  To allow tests and dsconf to manage indexes we
need the supporting objectClasses in place.

Fix Description:  This adds the Indexes
and Index types, as well as a linking method from a backend to get
it's related Indexes.

https://pagure.io/lib389/issue/59

Author: wibrown

Review by: ilias95, spichugi (Thanks!)

- - - - -
7831403a by Ilias Stamatis at 2017-10-11T15:24:25Z
Issue 67 - Add ensure_int function

Bug Description: get_attr_val_int("attr") method raises an Exception if
the requested attribute does not exist. All other similar methods return
None in that case.

Fix Description: Add a new ensure_int() utility function that checks
against None, in consistency with existing functions.

https://pagure.io/lib389/issue/67

Author: Ilias95

Review by: wibrown (Thank you very much!)

- - - - -
b2edcb29 by Ilias Stamatis at 2017-10-11T15:24:25Z
Issue31 - Add memberOf fix-up task

Description:
- Implement a new-style Task class based on DSLdapObject.
- Create the memberof fix-up task class.
- Add dsconf support for calling the task.
- Write a functional test for testing the fix-up task.
- Small bug fixes on the test utils for plugin tests.
- Replace some "from foo import *" imports with explicit imports.

https://pagure.io/lib389/issue/31

Author: Ilias95

Review by: wibrown (Thank you, great work!)

- - - - -
5881b22e by Ilias Stamatis at 2017-10-11T15:24:25Z
Issue 31 - Allow complete removal of some memberOf attrs

Description: #49284 is now fixed. Add code for complete removal of
memberOfEntryScope and memberOfEntryScopeExcludeSubtree attributes.
This was not possible before because of a bug in DS.

https://pagure.io/lib389/issue/31

Author: Ilias95

Review by: wibrown (Thanks!)

- - - - -
e7f66dbf by Ilias Stamatis at 2017-10-11T15:24:25Z
Issue 78 - Add exists() method to DSLdapObject

Description: Add an exists() method to DSLdapObject which returns
a boolean indicating if an entry exists or not. This can be useful in
cases such as when we want to check whether an entry exists in order
to create it if needed, or when checking for the completion of a task
(no task entry means that the task have been completed).

https://pagure.io/lib389/issue/78

Author: Ilias95

Review by: wibrown (thank you so much!)

- - - - -
fff949b7 by Ilias Stamatis at 2017-10-11T15:24:25Z
Issue 48 - Add support for USN plugin

Description: Add dsconf support for configuring the USN plugin from the
command line, the cleanup task for removing tombstone entries, and functional
tests for testing its functionality.

https://pagure.io/lib389/issue/48

Author: Ilias95

Review by: wibrown (Great work, thanks!)

- - - - -
bdf528bf by Simon Pichugin at 2017-10-11T15:24:25Z
Add PLUGIN_ADDN to _constants.py

Description: Add a constant for addn plugin. One line rule.

- - - - -
21ac0b9c by Ilias Stamatis at 2017-10-11T15:24:25Z
Issue 49 - Add support for whoami plugin

Description: Add dsconf support for enabling / disabling whoami plugin.

https://pagure.io/lib389/issue/49

Author: Ilias95

Review by: wibrown (Great work!)

- - - - -
82817e83 by Mark Reynolds at 2017-10-11T15:24:25Z
Ticket 87 - Update accesslog regec for HR etimes

Description:  Update the regex patterns to handle high resolution etimes.

https://pagure.io/lib389/issue/87

Reviewed by: firstyear(Thanks!)

- - - - -
f2c23982 by Simon Pichugin at 2017-10-11T15:24:25Z
Issue 83 - Add an util for generating instance parameters

Description: Currently, we create instance parameters (HOST_MASTER_1,
PORT_MASTER_1, etc.) in _constants.py module.
The commit adds generate_ds_params() to lib389.utils.

The util can generate a host, port, secure port, server ID and replica ID
for the selected role and instance number. I.e. inst_num=1, role="master".

Fix topologies.py module to use the new util.
Add create_topology() function.
Remove agmts from TopologyMain, we should use agreement module for that.

https://pagure.io/lib389/issue/83

Reviewed by: wibrown (Thanks!)

- - - - -
1913a0ae by Ilias Stamatis at 2017-10-11T15:24:25Z
Issue 86 - add build dir to gitignore

Description: Add the build dir that is created after running "make
build" to .gitignore.

https://pagure.io/lib389/issue/86

Author: Ilias95

Review by: wibrown (Thanks!)

- - - - -
94addd5a by Simon Pichugin at 2017-10-11T15:24:25Z
Issue 79 - Fix replica.py and add tests

Description: Move tests for ReplicaLegacy to replicaLegacy_test.py
module. Add a new test suite for Replicas(DSLdapObject).
Fix or change:
- Add enum34 to pip and python-enum34 to specfile for RHEL7/Fedora
- _constants.py:
  - replace REPLICAROLE_* with ReplicaRole(Enum) object
  - change the type of REPLICA_FLAGS_* from str to int
  - remove not used constants REPLICA_TYPE_* and REPLICA_FLAGS_CON
- Agreement.create() - if property is None, define the dict object;
- Changelog.list() - make it consistent and return empry list
  if nothing is found;
- Replica:
  - Replace hard coded values with variables if possible;
  - Fix get_role() functions so it will return right result;
  - Fix deleteAgreements. It now doesn't have suffix param;
  - Fix typos on variables;
  - Make promote() and demote() methods more explicit;
  - Rename Replicas.delete() to Replicas.disable because
    it coveres changelog and agreements deletion too;
  - Make docstring format consistent;

https://pagure.io/lib389/issue/79

Reviewed by: wibrown (Thanks!)

- - - - -
c5192ead by Mark Reynolds at 2017-10-11T15:24:25Z
Ticket 89 - Fix inconsistency with serverid

Description:  The path of least resistance is to use "standalone1"
              instead of "standalone_1".

https://pagure.io/lib389/issue/89

Reviewed by: mreynolds (one line commit rule)

- - - - -
eac24405 by Simon Pichugin at 2017-10-11T15:24:25Z
Issue 91 - Fix replication topology

Bug Description: When you use m1h1c1 topologies the replica type is
incorrectly set for consumers(maybe the others too), and it adds repl
agreements to consumers. Also there is circular repl initialization
going on which causes everybody to have different db generation IDs
which then breaks replication.

Fix Description: Refactor create_topology(), make it more readable
and depricate cascading replication scenarion. Let's create it when we
need it.
Fix an issue with creating agreements on consumers.
Fix an issue with agreements initialization.
Remade topology_m1h1c1 so it wouldn't use create_topology() func.
Fix small replica.py issues.

Reviewed by: wibrown, mreynolds (Thanks!)

https://pagure.io/lib389/issue/91

- - - - -
152473dd by Simon Pichugin at 2017-10-11T15:24:25Z
Issue 85 - Remove legacy replication attribute

Bug description: In 389-ds all references to legacy replication
has been removed (except in schema). But if in lib389 a replica
is created it always sets nsds5ReplicaLegacyConsumer: off.
This is no longer needed and confusing.

Fix description: Remove the properties related to legacy
replication attribute. Fix basedn of Replicas object.
Fix replicaLegacy tests.

Reviewed by: mreynolds (Thanks!)

https://pagure.io/lib389/issue/85

- - - - -
604bcdee by William Brown at 2017-10-11T15:24:26Z
Ticket 88 - python install and remove for tests

Bug Description:  We need to be able to test instances with python
and no perl tools. This will help us to progress and remove perl
from the codebase, helping improve our portability.

Fix Description:  Add support to remove instances with python.
Fix some setup python issues. Improve the lib389 test suite
to handle some of the edge cases acound the changes.

In general, this helps to improve our python 3 support across
the board as this allows us to perform pure python 3 installs
and tests.

https://pagure.io/lib389/issue/88

Author: wibrown

Review by: spichugi (Thank you!)

- - - - -
252edd21 by Ilias Stamatis at 2017-10-11T15:24:26Z
Issue 93 - Fix test cases in ctl_dbtasks_test.py

Description: There's a typo in ctl_dbtasks_test.py making 2 test cases
to fail.

https://pagure.io/lib389/issue/93

Author: Ilias95

Review by: spichugi, wibrown (Thanks!)

- - - - -
7f14842c by Ilias Stamatis at 2017-10-11T15:24:26Z
Issue 92 - display_attr() should return str not bytes in py3

Bug Description: display_attr method of mapped object returns
bytes in python3. Since we use this function to display display
data to the end user we don't want this.

There is also a bug in get_attr_vals_type() methods. These
methods call a non-existing function due to a typo.

Fix Description: Call get_attr_valus_utf8 instead of get_attr_vals
from within display_attr method. Also fix the bug with the
get_attr_vals_type by correcting the method's name.

https://pagure.io/lib389/issue/92

Author: Ilias95

Review by: wibrown (Thanks!)

- - - - -
b61b99a9 by Ilias Stamatis at 2017-10-11T15:24:26Z
Issue 74 - Advice users to set referint-update-delay to 0

Bug Description: Support for asynchronous referential integrity updates
might be dropped in the future and the referint-update-delay attribute
might become deprecated. We need to warn the users.

Fix Description: Display a warning when the healthcheck command is
issued and advise the user to set this attribute to 0.

https://pagure.io/lib389/issue/74

Author: Ilias95

Review by: wibrown (Thanks!)

- - - - -
f4c15476 by Ilias Stamatis at 2017-10-11T15:24:26Z
Issue 46 - dsconf support for dynamic schema reload

Description: Add dsconf support for dynamically reloading schema.

Notes: Renamed old-style Schema class to SchemaLegacy.

https://pagure.io/lib389/issue/46

Author: Ilias95

Review by: wibrown (thanks!)

- - - - -
e34b5cb9 by Ilias Stamatis at 2017-10-11T15:24:26Z
Issue 45 - Add support for Rootdn Access Control plugin

Description: Add dsconf support for configuring the Rootdn Access
Control plugin from the command line.

https://pagure.io/lib389/issue/45

Author: Ilias95

Review by: wibrown (Thanks!)

- - - - -
e1ce0261 by Ilias Stamatis at 2017-10-11T15:24:26Z
Issue 43 - Add support for Referential Integrity plugin

Description: Add dsconf support for configuring the Referential Integrity
plugin from the command line, along with functional tests for testing its
functionality.

https://pagure.io/lib389/issue/43

Author: Ilias95

Review by: wibrown (Thanks!)

- - - - -
4e51f455 by Simon Pichugin at 2017-10-11T15:24:26Z
Issue #77 - Add sphinx documentation

Description: Add the main documentation part formatted in rST.
Add sphinx directory structure and Makefile.
Documentation in HTML format can be generated with 'make html'.

https://pagure.io/lib389/issue/77

Reviewed by: mreynolds (Thanks!)

- - - - -
5bf84f94 by Viktor Ashirov at 2017-10-11T15:24:26Z
Issue 96 - Change binaries' names

Bug Description:
After https://pagure.io/389-ds-base/issue/49348 binaries dropped
their wrappers and -bin from their names. In lib389 we called them
directly, instead of using wrappers.

Fix Description:
Change binaries' names

https://pagure.io/lib389/issue/96

Reviewed by: mreynolds (Thanks!)

- - - - -
1989c0ee by Simon Pichugin at 2017-10-11T15:24:26Z
Issue #77 - Refactor docstrings in rST format - part 1

Decsription: With the first part, refactor docstrings in modules:
_mapped_object.py, aci.py, agreement.py, backend.py,
changelog.py, mappingTree.py, replica.py, repltools.py

Format it properly and put to index.rst

Add a script for links to the code in the docs.

https://pagure.io/lib389/issue/77

Reviewed by: wibrown (Thanks!)

- - - - -
4700e8b1 by Simon Pichugin at 2017-10-11T15:24:26Z
Issue #77 - Fix changelogdb param issue

Bug description: self.dbdir used before an assignment
in Changelog(conn) __init__ part, while self.dbdir is defined
in __initPart2 of DirSrv object.

Fix description: Add self.changelogdb to __initPart2 with default value.
Assign a new value in conn.changelog.create().

https://pagure.io/lib389/issue/77

Reviewed by: mreynolds (Thanks!)

- - - - -
4cb5bd7e by Sankar Ramalingam at 2017-10-11T15:24:26Z
Issue #98 - Fix dbscan output

Description: Dbscan output is printing new line characters and tab
characters. So, I changed the subprocess.Popen to subprocess.check_
output to return the exact the same output as command line.

https://pagure.io/lib389/issue/98

Reviewed by: spichugi

- - - - -
3da710f2 by Simon Pichugin at 2017-10-11T15:24:27Z
Issue 99 - Fix typo in create_topology

Description: We need to use the roles from the 'for' loop while creating
agreements. If we don't, we will have failures while creating 'asymmetric'
setups like two masters and one consumer.

https://pagure.io/lib389/issue/99

Reviewed by: amsharma, sramling (Thanks!)

- - - - -
7d5154f6 by William Brown at 2017-10-11T15:24:27Z
Ticket 102 - referral support

Bug Description:  We need support for smart referrals to test
managedsait controls.

Fix Description:  Add support for referral objects and a test to
demonstrate they work.

https://pagure.io/lib389/issue/102

Author: wibrown

Review by: spichugi (Thank you!)

- - - - -
3b56c4d6 by alisha17 at 2017-10-11T15:24:27Z
Ticket 101 - BaseException.message has been deprecated in Python3

Bug Description: Error is caused when some exception is raised in
dsidm file becuase BaseException.message has been deprecated in
Python3

Fix Description:  Using str(Exception) fixes the bug for dsidm
as well as all other files using BaseException.message

https://pagure.io/lib389/issue/101

Author: Alisha Aneja

Review by: wibrown (Thanks for your patch!)

- - - - -
9dccfea3 by William Brown at 2017-10-12T06:40:56Z
Ticket 49363 - Merge lib389

Bug Description:  The issue is that we have a split: we have tests in
389-ds-base/dirsrvtests that are often version dependent. They relate
to features of the server, or issues in specific versions of the server
that may not exist in older versions. Today we kind of stradle the line
of "it's a bit of both". We have tests in 389-ds-base that depend on
versions of lib389 - but lib389 moves quickly and has little ability to
distinguish 389-ds-base versions.

Fix Description:  Merge lib389 source code and update related build components.

https://pagure.io/389-ds-base/issue/49363

Author: wibrown

Review by: wibrown (thanks everyone!)

- - - - -
7b36a268 by Amita Sharma at 2017-10-12T10:39:10Z
Issue 49381 - Refactor ACL test suite docstrings

Description: We need to have properly formatted and detailed
docstrings in all existing test suites. This commit updates ACL
test suit docstrings.

https://pagure.io/389-ds-base/issue/49381

Reviewed by: Simon Pichugin

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
7eee594f by Viktor Ashirov at 2017-10-12T11:46:02Z
Issue 49400 - Add clang support to rpm builds

Bug Description:
Server falis to build using clang because we need to explicitly link to
libatomic.

Fix Description:
Explicitly link to libatomic.
Add support for clang in spec file.

https://pagure.io/389-ds-base/issue/49400

Reviewed by: mreynolds (Thanks!)

- - - - -
2b8b6db1 by alisha17 at 2017-10-13T10:37:15Z
Ticket 103 - Make sysconfig where it is expected to exist

Bug Description:  Error when creating new instance
while opening sysconfig because sysconfig doesn't
exists

Fix Description:  Create a sysconfig directory before
opening it

https://pagure.io/lib389/issue/103

Author: Alisha Aneja

Review by: William Brown

- - - - -
df4492b1 by Mark Reynolds at 2017-10-13T11:49:28Z
Ticket 48235 - Remove memberOf global lock

Bug Description:  The memberOf global lock no longer servers a purpose since
                  the plugin is BETXN.  This was causing potential deadlocks
                  when multiple backends are used.

Fix Description:  Remove the lock, and rework the fixup/ancestors caches/hashtables.
                  Instead of reusing a single cache, we create a fresh cache
                  when we copy the plugin config (which only happens at the start
                  of an operation).  Then we destroy the caches when we free
                  the config.

https://pagure.io/389-ds-base/issue/48235

Reviewed by: firstyear & tbordaz(Thanks!!)

(cherry picked from commit 184b8a164f4ed456c72d58038aa9a0d512be61fa)
(cherry picked from commit 8915d8d87051b5415f7c07603c378d9310b7c783)

- - - - -
88fac273 by Amita Sharma at 2017-10-13T14:11:03Z
Issue 49381 - Refactor filter test suite docstrings

Description: We need to have properly formatted and detailed
docstrings in all existing test suites. Filter test suite is
modified in this commit.

https://pagure.io/389-ds-base/issue/49381

Reviewed by: Simon

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
6b6878b0 by William Brown at 2017-10-14T08:57:30Z
Ticket 49403 - tidy ns logging

Bug Description:  nunc-stans logging would swap to a printf
mode if DEBUG was defined.

Fix Description:  NS is in a good state, so we can use slapd log
instead.

https://pagure.io/389-ds-base/issue/49403

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
dcba969f by Thierry Bordaz at 2017-10-16T11:55:30Z
Ticket 49394 - slapi_pblock_get may leave unchanged the provided variable

Bug Description:
	Since 1.3.6.4 the pblock struct is a split in sub-structs
	(https://pagure.io/389-ds-base/issue/49097)

	Before, it was a quite flat calloc struct and any slapi-pblock-get
	retrieved the field (NULL if not previously slapi_pblock_set) and
	assigned the provided variable.

	Now, the sub-struct are allocated on demand (slapi_pblock_set).
	If a substruct that contains the requested field is not allocated the
	provided variable is unchanged.

	This is a change of behavior, because a uninitialized local variable can
	get random value (stack) if the lookup field/struct has not been set.

Fix Description:
	Update slapi_pblock_set so that it systematically sets the
	provided variable when those substructs are NULL
		pb_mr
		pb_dse
		pb_task
		pb_misc
		pb_intop
		pb_intplugin
		pb_deprecated

https://pagure.io/389-ds-base/issue/49394

Reviewed by: Mark Reynolds, William Brown

Platforms tested: F23

Flag Day: no

Doc impact: no

- - - - -
1e28c75c by Simon Pichugin at 2017-10-16T12:56:37Z
Issue 49381 - Refactor numerous suite docstrings - Part 2

Description: We need to have properly formatted and detailed
docstrings in all existing test suites.
Suites that were changed in this commit:
sasl allowed_mechs, basic, password, replication.
Also, fix create test.py tool, create password policy fixture in
syntax test, in replication move get_repl_entries to __init__.py,
move test_double_delete, test_password_repl_error, test_invalid_agmt
from acceptance_test.py to regression_test.py.
Rename plain.py to plain_test.py
and allowed_mechs.py to allowed_mechs_test.py.

https://pagure.io/389-ds-base/issue/49381

Reviewed by: amsharma, vashirov, wibrown (Thanks!)

- - - - -
34a184a9 by Thierry Bordaz at 2017-10-16T13:51:02Z
Ticket 49394 - build warning

Bug Description:
	Fixing an invalid type assignment

- - - - -
3eb443b0 by Mark Reynolds at 2017-10-16T14:44:29Z
Ticket 48235 - remove memberof lock (cherry-pick error)

Description: Fix cherry-pick error

https://pagure.io/389-ds-base/issue/48235

Reviewed by: mreynolds(one line commit rule)

- - - - -
bc6dbf15 by Mark Reynolds at 2017-10-16T20:18:32Z
Ticket 49402 - Adding a database entry with the same database name that was deleted hangs server at shutdown

Bug Description:  At shutdown, after a backend was deleted, which also had a import
                  task run, the server hangs at shutdown.  The issue is that the
                  import task destructor used the ldbm inst struct to see if it was
                  busy, but the inst was freed and the destructor was checking invalid
                  memory which caused a false positive on the "busy" check.

Fix Description:  Do not check if the instance is busy to tell if it's okay to remove
                  the task, instead just check the task's state.

https://pagure.io/389-ds-base/issue/49402

Reviewed by: lkrispen(Thanks!)

- - - - -
e8c56aaf by William Brown at 2017-10-17T14:18:14Z
Ticket lib389 3 - python 3 support

Bug Description:  Replication tests did not support python 3

Fix Description:  Improve test running capability for python 3 including
fixing master init.

https://pagure.io/lib389/issue/3

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
c973e639 by Thierry Bordaz at 2017-10-18T12:20:51Z
Ticket 49064 - RFE allow to enable MemberOf plugin in dedicated consumer

Bug Description:
	memberof triggers some internal updates to add/del 'memberof' values.
	on a readonly consumer, those updates selects a REFERRAL_ON_UPDATE backend
	and that is not followed by internal updates.
	At the end of the day, the update is rejected and if memberof plugin is enabled
	replication will stuck on that rejected update

Fix Description:
	internal updates from memberof need to bypassing referrals.
	So they flag internal updates SLAPI_OP_FLAG_BYPASS_REFERRALS, so that mtn_get_be
	(mapping tree selection) will not return the referrals.

https://pagure.io/389-ds-base/issue/49064

Reviewed by: Ludwig Krispenz, William Brown (thanks a LOT !)

Platforms tested: F23 (all tickets + basic suite)

Flag Day: no

Doc impact: no

- - - - -
478a2aeb by Thierry Bordaz at 2017-10-18T14:38:56Z
Ticket 49064 - testcase hardening

- - - - -
e123acb6 by Mark Reynolds at 2017-10-18T20:08:25Z
Ticket 48006 - Missing warning for invalid replica backoff
 configuration

Description:  Add warning if you try to set a min backoff time that is
              greater than the configured maximum, or the max time that
              is less than the minimum.

              Also fixed compiler warning in ldbm_config.c

https://pagure.io/389-ds-base/issue/48006

Reviewed by: firstyear(Thanks!)

- - - - -
35c20caf by Thierry Bordaz at 2017-10-19T12:18:52Z
Ticket 49386 - Memberof should be ignore MODRDN when the pre/post entry are identical

Bug Description:
	If a modrdn command renames an entry in itself it is inexpensive for the
	core server but can be expensive for memberof.
	member will update all entries referring (memberof value) the target entry.
	The updates will delete and add back the target entry DN.

Fix Description:
	In the early step of memberof modrdn callback, checks if pre_sdn and post_sdn
	are identical.
	If they are it skips the update

https://pagure.io/389-ds-base/issue/49386

Reviewed by: Mark Reynolds

Platforms tested: F23

Flag Day: no

Doc impact: no

foo

- - - - -
e46749b7 by Mark Reynolds at 2017-10-19T18:44:38Z
Ticket 48681 - Use of uninitialized value in string ne at /usr/bin/logconv.pl line 2565, <$LOGFH> line 4

Bug description:  The original fix for 48681 added a regression in regards to perl
                  warning everytime you ran the script.  That was due to a new hash
                  for sasl binds that was not initialized.

Fix Description:  Check is the saslbind hash "exists" before checking its value.

https://pagure.io/389-ds-base/issue/48681

Reviewed by: mreynolds (one line fix)

- - - - -
45d2fd4b by Mark Reynolds at 2017-10-20T15:11:21Z
Ticket 49407 - status-dirsrv shows ellipsed lines

Bug Description:  To show the full output you have to pass "-l" to systemctl,
                  but there is no way to use this option with the current design.

Fix Description:  Just show the full lines by default, as adding options can break
                  the script's current usage.

https://pagure.io/389-ds-base/issue/49407

Reviewed by: tbordaz(Thanks!)

- - - - -
296f0abb by Mark Reynolds at 2017-10-20T15:52:54Z
Ticket 49408 - Server allows to set any nsds5replicaid in the existing replica entry

Description:  There was no value validation for replica ID.  Now there is.

https://pagure.io/389-ds-base/issue/49408

Reviewed by: tbordaz(Thanks!)

- - - - -
63a0a59c by Mark Reynolds at 2017-10-20T16:12:41Z
Ticket 49374 -  server fails to start because maxdisksize is recognized incorrectly

Bug Description:  When directly editting dse.ldif, the server had a check
                  when setting the log maxdiskspace vs maxlogsize.  If the
                  maxlogsize is processed first and it is higher than the
                  default maxdisksspace then it throw an error and the server
                  fails to start.

                  If you attempt this same operation using ldapmodify it
                  works as "live" updates check all the mods first, so the
                  order of the attributes does not matter.

Fix description:  Remove the size checks from the attribute set function.
                  It is technically redundant since it is correctly checked
                  by the configdse code.

https://pagure.io/389-ds-base/issue/49374

Reviewed by: tbordaz(Thanks!)

- - - - -
a43a8efc by Mohammad Nweider at 2017-10-25T00:14:08Z
Ticket 49401 - improve valueset sorted performance on delete

Bug Description:  valueset sorted maintains a list of syntax sorted
references to the attributes of the entry. During addition these are
modified and added properly, so they stay sorted.

However, in the past to maintain the sorted property, during a delete
we would need to remove the vs->sorted array, and recreate it via qsort,

While this was an improvement from past (where we would removed
vs->sorted during an attr delete), it still has performance implications
on very very large datasets, IE 50,000 member groups with
addition/deletion, large entry caches and replication.

Fix Description:  Implement a new algorithm that is able to maintain
existing sort data in a near linear time.

https://pagure.io/389-ds-base/issue/49401

Author: nweiderm, wibrown

Review by: wibrown, lkrispen, tbordaz (Thanks nweiderm!)

- - - - -
b0b63461 by Viktor Ashirov at 2017-10-25T05:31:38Z
Issue 49409 - Update lib389 requirements

Bug Description:
requirements.txt and setup.py are out of date.

Fix Description:
Add required dependencies, update tox.ini.
Remove setup.cfg because it's not needed anymore (we use our own spec
file instead of one generated by bdist_rpm).

https://pagure.io/389-ds-base/issue/49409

Reviewed by: wibrown (Thanks!)

- - - - -
6098e7b9 by Mark Reynolds at 2017-10-25T14:56:04Z
Ticket 48681 - Use of uninitialized value in string ne at /usr/bin/logconv.pl

Bug Description:  ldapi connections were not properly porcessed by the
                  connection parsing code which lead to uninitialized errors.

Fix Description:  Modify the connection IP address regex's to include "local"

https://pagure.io/389-ds-base/issue/48681

Reviewd by: mreynolds (one line commit rule)

- - - - -
52ba2aba by Mohammad Nweider at 2017-10-25T17:53:30Z
Ticket 49401 - Fix compiler incompatible-pointer-types warnings

Bug Description: vs->sorted was integer pointer in older versions,
    but now it's size_t pointer, this is causing compiler warnings
    during the build

Fix Description: use size_t pointers instead of integer pointers for vs->sorted and sorted2

Review By: mreynolds

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
2086d052 by Mark Reynolds at 2017-10-26T14:21:35Z
Ticket 48894 - harden valueset_array_to_sorted_quick valueset
 access

Description:  It's possible during the sorting of a valueset to access an
              array element past the allocated size, and also go below the index 0.

https://pagure.io/389-ds-base/issue/48894

Reviewed by: nweiderm (Thanks!)

- - - - -
ba0ea20d by Mark Reynolds at 2017-10-31T01:23:56Z
Ticket 49432 - Add complex fliter CI test

Description:  Add a CI test for complex filters

https://pagure.io/389-ds-base/issue/49432

Reviewed by: wibrown (Thanks Mark!)

- - - - -
5c89dd8f by William Brown at 2017-10-31T01:24:03Z
Ticket 49432 - filter optimise crash

Bug Description:  In a certain condition with a filter, when we
removed the equality candidate to optimise it, with a nested
and, during the merge process we would segfault

Fix Description:  Fix the merge subfilter process to be cleaner
and work in all conditions. Merge the set of filter tests to
cmocka in addition to the python tests to help catch this earlier

https://pagure.io/389-ds-base/issue/49432

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
f4d86bb9 by Viktor Ashirov at 2017-10-31T11:24:26Z
Issue 49434 - RPM build errors

Bug Description:
When rpm is built on epel-7 (using mock), build fails because
python libraries and tests are installed, but not packaged.

Fix Description:
Don't install files if we don't plan to package them.

https://pagure.io/389-ds-base/issue/49434

Reviewed by: mreynolds (Thanks!)

- - - - -
6ef4eb5a by William Brown at 2017-11-01T02:27:24Z
Ticket 3 - python 3 support - filter test

Bug Description:  Filter suite did not work with python 3

Fix Description:  Fix issue in default acis during configuration
and fix tests to work with python 3

https://pagure.io/lib389/issue/3

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
e3d4a52e by Amita Sharma at 2017-11-01T22:46:34Z
Issue lib389 3 - Python 3 support for pwdPolicy_controls_test.py

Bug Description: password/pwdPolicy_controls_test.py tests did not support python 3

Fix Description: use byte type for the LDAP values and fix LDAPError

https://pagure.io/lib389/issue/3

Reviewed by: William Brown (Thanks)

- - - - -
75144644 by William Brown at 2017-11-02T02:35:36Z
Ticket 49424 - Resolve csiphash alignment issues

Bug Description:  On some platforms, uint64_t is not the same size
as a void * - as well, if the input is not aligned correctly, then
a number of nasty crashes can result

Fix Description:  Instead of relying on alignment to be correct,
we should memcpy the data to inputs instead.

https://pagure.io/389-ds-base/issue/49424

Author: wibrown

Review by: lslebodn, cgrzemba, vashirov, mreynolds (Thanks!)

- - - - -
03df576c by Mark Reynolds at 2017-11-02T12:23:37Z
Ticket 48007 - CI test to test changelog trimming interval

Description:  CI test to verify that changes to the trimming interval
              takes effect immediately

https://pagure.io/389-ds-base/issue/48007

Reviewed by: firstyear(Thanks!)

- - - - -
ee4428a3 by William Brown at 2017-11-03T00:07:50Z
Ticket 49436 - double free in COS in some conditions

Bug Description:  virtualattrs and COS have some serious memory
ownership issues. What was happening is that COS with multiple
attributes using the same sp_handle would cause a structure
to be registered twice. During shutdown we would then trigger
a double free in the process.

Fix Description:  Change the behaviour of sp_handles to use a
handle *per* attribute we register to guarantee the assocation
between them.

https://pagure.io/389-ds-base/issue/49436

Author: wibrown

Review by: mreynolds, vashirov (Thanks!)

- - - - -
9c4a3150 by William Brown at 2017-11-03T00:23:23Z
Ticket 103 - sysconfig not found

Bug Description:  In a prefix build we don't have a /etc/sysconfig
file, so we need to create it. However, if we try to create it
multiple times python will raise an exception.

Fix Description:  Catch and ignore the exception if the directory
already exists.

https://pagure.io/lib389/issue/103

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
fe859456 by Simon Pichugin at 2017-11-03T12:28:31Z
Issue lib389 3 - Python 3 support for ACL test suite

Bug Description: acl/acl_test.py tests did not support python 3

Fix Description: Add organisationalrole.py module to lib389/idm.
Improve acl/acl_test.py to use DSLdapObjects and byte types.
Fix some docstrings. Remove test_mode_switch_default_to_legacy
and add it's functionality to other 'legacy' tests so we can
run each test as standalone.

https://pagure.io/lib389/issue/3

Reviewed by: vashirov, wibrown (Thanks!)

- - - - -
f6b0e184 by Mark Reynolds at 2017-11-03T13:32:19Z
Ticket 48393 - Improve replication config validation

Bug Description:  There was inconsistent behavior when modifying and adding replication
                  configurations and agreements.  There were also a few places where
                  unsigned ints were used for values which made checking for negative
                  values impossible.

Fix Description:  Added a new function to properly check "number" attribute values.
                  Also forced failure on the actual update if an invalid value was used
                  (previously we would log an error and use some default value).  Also
                  made all the int types consistent.

https://pagure.io/389-ds-base/issue/48393

Reviewed by: firstyear(Thanks!)

- - - - -
e1f866a5 by Mark Reynolds at 2017-11-03T14:57:09Z
Ticket 49439 - cleanallruv is not logging information

Bug Description:  During the logging refector effro from ticket 48978
                  a mistake was made and cleanruv_log() was using
                  LOG_NOTICE (which is not a true log level), it was
                  supposed to be SLAPI_LOG_NOTICE.

                  We also use DEBUG defines to contorl the logging for
                  debug builds

Fix Description:  Remove the LDAP_DEBUG defines in cleanruv_log, and set
                  the correct logging severity level.

https://pagure.io/389-ds-base/issue/49439

Reviewed by: firstyear(Thanks!)

- - - - -
43164703 by Mark Reynolds at 2017-11-03T16:18:26Z
Ticket 48393 - fix copy and paste error

Description:  Copy and paste error when validating repl agmt

https://pagure.io/389-ds-base/issue/48393

Reviewed by: mreynolds(one line commit rule)

- - - - -
e622d95e by Mark Reynolds at 2017-11-03T17:14:11Z
Bump version to 1.4.0.2

- - - - -
11974a08 by William Brown at 2017-11-06T22:13:56Z
Ticket 49435 - Fix NS race condition on loaded test systems

Bug Description:  During a test run, on a heavily loaded systems
some events would time out before they could occur correctly.

Fix Description:  Change the structure of events to mitigate
a deref performance hit, and add a ns_job_wait conditional
that allows blocking on a job to complete so that tests do not
require time based checks.

https://pagure.io/389-ds-base/issue/49435

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
77df9a54 by William Brown at 2017-11-07T02:23:33Z
Ticket 17 - lib389 - dsremove support

Bug Description:  We need to support clean removal of our server
instances with pure python cli and test cases

Fix Description:  This fixes support for removing replicated
instances wth python, and adds support for dsctl <inst> remove.

https://pagure.io/lib389/issue/17

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
b4497c4f by Mark Reynolds at 2017-11-07T03:14:06Z
Ticket 49441 - Import crashes with large indexed binary
 attributes

Bug Description:  Importing an ldif file that contains entries with large
                  binary attributes that are indexed crashes the server.
                  The crash occurs when "encoding" the binary value to a
                  string for debug logging, where we "underflow" the buffer
                  space index which then allows the string buffer to overflow.

Fix Description:  While filling the string buffer with the encoded binary
                  value we need to make sure if the buffer space is greater
                  than zero before decrementing it.

                  Also check if trace logging is being used before we actually
                  call the logging function which calls the "encoded" function
                  first.  This way we avoid this costly "encoding" on every
                  index call we make.

https://pagure.io/389-ds-base/issue/49441

Reviewed by: firstyear(Thanks!)

- - - - -
b3629af0 by William Brown at 2017-11-07T06:16:55Z
Ticket 49377 - Incoming BER too large with TLS on plain port

Bug Description:  When doing TLS to a plain port, a message of
"ber element 3 bytes too large for max ber" when max ber > 3.

Fix Description:  When ber_len < maxber, report that the request
may be misformed instead of "oversize" instead. This can lead
to a better diagnosis.

https://pagure.io/389-ds-base/issue/49377

Author: wibrown

Review by: mreynolds (thanks!)

- - - - -
be4d7e5a by William Brown at 2017-11-07T07:09:18Z
Ticket 49441 - Import crashes - oneline fix

Bug Description:  index.c fails to compile.

Fix Description:  Excess braces due to copy paste issue.

https://pagure.io/389-ds-base/issue/49441

Author: wibrown

Review by: oneline rule

- - - - -
c0a7be7c by Thierry Bordaz at 2017-11-07T13:38:17Z
Ticket 49412 - SIGSEV when setting invalid changelog config value

Bug Description:
	If admin configures attribute of "cn=changelog5,cn=config" with
	empty value, it can trigger a sigsev

Fix Description:
	Testing empty value

https://pagure.io/389-ds-base/issue/49412

Reviewed by: Ludwig Krispenz, Mark Reynolds (thanks to you)

Platforms tested: F23

Flag Day: no

Doc impact: no

- - - - -
9de5d88e by William Brown at 2017-11-08T02:51:21Z
Ticket lib389 3 - python 3 support for betxn test

Bug Description:  Support python 3 for betxn test.

Fix Description:  python 3 support for betxn test, including
addition of rename supprot for mapped objects.

https://pagure.io/lib389/issue/3

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
8ab0614e by William Brown at 2017-11-09T03:56:17Z
Ticket 3 lib389 - python 3 support ds_logs tests

Bug Description:  Support python  3 for the ds logs tests

Fix Description:  Fix the logs test to use DSLdapObjects correctly.
Change the doc string on DSLdapObjects.get to reflect it's use
with "*", as well as correct how it searches attrs.

https://pagure.io/lib389/issue/3

Author: wibrown

Review by: spichugi

- - - - -
7f899a45 by William Brown at 2017-11-09T09:24:55Z
Ticket 3 - lib389 - config test

Bug Description:  python 3 suppor for config test

Fix Description:  Fix python 3 support for config test, correct
a small issue in removal, and improve some logging of large values

https://pagure.io/lib389/issue/3

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
b0689cd2 by Ludwig Krispenz at 2017-11-09T10:28:34Z
    Ticket 48118 - At startup, changelog can be erronously rebuilt after a normal shutdown

    Problem: There are two problems that can lead to inconsistent database and changelog maxruv:
    1] the database ruv is written periodically in th ehouskeeping thread and at shutdown. It
       relies on teh ruv_dirty flag, but due to a race condition this can be reset befor writing
       the ruv
    2] the changelog max ruv is updated whenever an operation is commutted, but in case of internal
       operations inside the txn for a client operation, if the operation is aborted the cl maxruv
       is not reset. Since it is only written at shutdown this normally is no problem, but if the
       aborted operation is the last before shutdown or is aborted by shutdown the cl ruv is incorrect

    Fix: the fix is in two parts:
    1] remove the use of the dirty flag, ensure that the ruv is always written. The overhead for writing
       a database ruv that has not changed is minimal
    2] when writing the changelog maxruv check if the macsns it contains are really present in the
       changelog. If not the maxruv is not written, it will be reconstructed at the next startup

    Reviewed by: William,Thierry - Thanks

- - - - -
c917b93a by Ludwig Krispenz at 2017-11-09T14:34:42Z
Ticket 49443 - scope one searches in 1.3.7 give incorrect results

Bug: if a onelevel search is done for an unidexed attribute, the filter test is skipped
     and all children of the search base are returned

Fix: enforce filter test if allids

Reviewed by: Mark, thanks

- - - - -
f02cbddf by Amita Sharma at 2017-11-13T08:39:15Z
Ticket - lib389 - Python 3 support for memberof plugin test suit

Bug Description: ticket49064_test.py tests did not support python 3

Fix Description: Move the test file from tickets to suits/memberof_plugin/regression_test.py,
fix config_memberof and other methods to use DSLdapObject,
create test users and test groups using DSLdapObject in test case,
fix some docstrings and add more comments. Remove add_user,
add_group, update_member and memberof_fixup_task methods.
Remove ticket49064_test.py from dirsrvtests/tests/tickets/

https://pagure.io/lib389/issue/3

Reviewed by: William Brown (Huge Thanks!), spichugi

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
6ec27bc8 by William Brown at 2017-11-14T01:06:46Z
Ticket 49325 - Proof of concept rust tqueue in sds

Bug Description:  Rust is a modern systems programming language
in the style of C and C++. It has strict compile time guarantees
to the correctness of applications, and promises the potential
to reduce many times of security and stability issues including
bounds checking, null dereferences, use-after free and more.
This is achieved without a run time, instead using compile time
ownership and lifetime checks.

Fix Description:  This ticket is to add a proof of concept that
we can use Rust as an FFI language with existing C components.
This adds an optional configure argument to enable a rust thread
safe queue which is used by nunc-stans for event dispatch.

My tests already show it is safe (it passes all the existing test)
and the server when built with this option passes the basic suite.

Importantly it shows how we can integrate cargo with autotools,
and how to expose and C compatible apis from rust.

To use this, at configure time add "--enable-rust".

There are no other changes to the server without this flag, and it
is not a requirement to build DS, it's optional.

https://pagure.io/389-ds-base/issue/49325

Author: wibrown

Review by: mreynolds, tbordaz (Thank you!)

- - - - -
9e4b3815 by William Brown at 2017-11-14T03:22:02Z
Ticket 3 lib389 - python 3 compat for paged results test

Bug Description:  python 3 support for paged results test

Fix Description:  Rework paged results to use lib389 types. Add
sort controls to lib389. Fix rawaci on entry to return bytes. Add
support for organisation objects. Improve mappingtree to support
parent additions.

https://pagure.io/lib389/issue/3

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
4e8bf06a by Viktor Ashirov at 2017-11-14T09:08:16Z
Issue 49451 - Add environment markers to lib389 dependencies

Bug Description:
lib389 should be installable both on python2 and python3.
Currently it can be installed using python3 only because
of the python3-only dependencies.

Fix Description:
Use PEP-508 environment markers for setup.py and requirements.txt.

https://pagure.io/389-ds-base/issue/49451

Reviewed by: wibrown (Thanks!)

- - - - -
bdcbf5bf by Ludwig Krispenz at 2017-11-15T12:17:00Z
Ticket 48118 - fix compiler warning for incorrect return type

- - - - -
1b2a2d6c by Thierry Bordaz at 2017-11-15T15:25:41Z
Ticket 49410 - opened connection can remain no longer poll, like hanging

Bug Description:
	Some opened connection are no longer poll.
	Those connections has 'gettingber' toggle set although there is
	no more worker thread reading it.
	The reason they have gettingber set is that the last
	operation had 'persistent search' flag. With such flag
	gettingber is not reset.
	persistent flag is set even when no persistent search/sync_repl
	was received on the connection.
	The problem is that the flag is tested on the wrong operation.
	The tested operation can be
		- the first operation when the connection entered in turbo mode
		- the previous operation if several ops PDUs were read on the network
		- accessing random memory

	In theory testing the flag can lead to sigsev even
	if it never crash

Fix Description:
	The fix is to use the operation that is in the pblock
	In such case pb_op is no longer used, so we can get rid of it.
	In addition make pb_conn a local variable where it is used

https://pagure.io/389-ds-base/issue/49410

Reviewed by: Ludwig Krispenz, Mark Reynolds

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
a60ee097 by William Brown at 2017-11-15T23:04:12Z
Ticket 49427 - whitespace in fedse.c

Bug Description:  During our meeting in Brno, we noticed that our
"whitespace" in fedse.c needed an update. Do so accordingly!

Fix Description:  Add required elements.

https://pagure.io/389-ds-base/issue/49427

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
955510cd by William Brown at 2017-11-15T23:05:28Z
Ticket 49453 - passwd.py to use pwdhash defaults.

Bug Description:  pwdhash now uses the defaults from libslapd,
so we do not need to code this into passwd.py

Fix Description:  Change the default args for passwd.py to
simplify the call.

https://pagure.io/389-ds-base/issue/49453

Author: wibrown

Review by: alisha17, spichugi (Thank you!)

- - - - -
5493b868 by Simon Pichugin at 2017-11-16T13:26:47Z
Issue lib389 #77 - Refactor docstrings in rST format - part 2

Description: Improve docstring coverage for IDM modules.
Fix small typos in replica, backend and mappingTree modules.
Add Python 3 support chapter to guidelines.rst.
Fix links to the source code because of the lib389 repo merging.

https://pagure.io/lib389/issue/77

Reviewed by: mreynolds, wibrown (Thanks!)

- - - - -
6e794a8e by Mark Reynolds at 2017-11-16T18:27:42Z
Ticket 49454 - SSL Client Authentication breaks in FIPS mode

Bug Description:  Replication using SSL Client Auth breaks when FIPS
                  is enabled.  This is because FIPS mode changes the
                  internal certificate token name.

Fix Description:  If FIPS is enabled grab the token name from the internal
                  slot instead of using the default hardcoded internal
                  token name.

https://pagure.io/389-ds-base/issue/49454

Reviewed by: firstyear(Thanks!)

- - - - -
75e55e26 by William Brown at 2017-11-16T18:59:39Z
Ticket 49298 - Correct error codes with config restore.

Bug Description:  The piece of code uses 0 as an error - not 1,
and in some cases did not even check the codes or use the
correct logic.

Fix Description:  Cleanup dse_check_file to better check the
content of files and communicate issues to the admin. Correct
slapd_bootstrap_config to correctly handle the cases of removal
and restore.

https://pagure.io/389-ds-base/issue/49298

Author: wibrown

Review by: mreynoolds & spichugi

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
d40d9196 by William Brown at 2017-11-17T01:43:36Z
Ticket 49298 - fix complier warn

Bug Description:  Extra argument to error log in dse.c

Fix Description:  Remove extra argument.

https://pagure.io/389-ds-base/issue/49298

Author: wibrown

Review by: oneline rule.

- - - - -
25e62b69 by William Brown at 2017-11-17T01:47:26Z
Ticket 49448 - dynamic default pw scheme based on environment.

Bug Description:  In some cases the hardcoded default pw scheme
is not available, IE FIPS mode doesn't support pbkdf2. In this
case we need to support returning an appropriate scheme based
on environment.

Fix Description:  Add a new scheme name called "DEFAULT" with
a mechanic that pw_name2scheme will use this with environment
information to return an appropriate scheme.

https://pagure.io/389-ds-base/issue/49448

Author: wibrown

Review by: mreynolds, spichugi (Thanks!)

- - - - -
5c624198 by Christian Heimes at 2017-11-20T11:25:53Z
Fix spal_meminfo_get function prototype

Fixes RHBZ #1515191

Signed-off-by: Christian Heimes <cheimes at redhat.com>

- - - - -
691b1444 by Amita Sharma at 2017-11-20T11:57:25Z
Ticket 49455- Add tests to monitor test suit.

Description: Add test cases for monitor test suit
Add methods in lib389 monitor.py to support test cases

https://fedorahosted.org/389/ticket/49455

Reviewed by: William Brown

- - - - -
2ddfb06f by Mark Reynolds at 2017-11-20T15:40:08Z
Merge #49457 `Fix spal_meminfo_get function prototype`
- - - - -
8fc6d0a5 by Mark Reynolds at 2017-11-20T15:43:21Z
Bump version to 1.4.0.3

- - - - -
4c4d3c13 by Simon Pichugin at 2017-11-21T14:17:14Z
Issue 49408 - Add a test case for nsds5ReplicaId checks

Description: We need to check that nsDS5ReplicaId accepts
only valid values and rejects invalid ones.

https://pagure.io/389-ds-base/issue/49408

Reviewed by: wibrown (Thanks!)

- - - - -
545b627a by William Brown at 2017-11-21T16:06:51Z
Ticket 49298 - issue with test case and remove-ds.pl

Bug Description:  In some cases, the missing dse.ldif would
cause remove ds.pl to fail.

Fix Description:  Move and restore the dse.ldif instead to allow
the test to pass.

https://pagure.io/389-ds-base/issue/49298

Author: wibrown

Review by: vashirov (Thanks!)

- - - - -
22a11d99 by Thierry Bordaz at 2017-11-22T07:52:16Z
Ticket 49460 - replica_write_ruv log a failure even when it succeeds

Bug Description:
	Minor issue
	If the update of the DB RUV returns a success LDAP_SUCCESS (internal modify),
	it however logs an error as if it failed

	side effect of https://pagure.io/389-ds-base/issue/48118

Fix Description:
	Log a message only on failure

https://pagure.io/389-ds-base/issue/49460

Reviewed by: Ludwig Krispenz, William Brown

Platforms tested: F23

Flag Day: no

Doc impact: no

- - - - -
a4062055 by William Brown at 2017-11-22T09:45:54Z
Ticket 49444 - huaf in task.c during high load import

Bug Description:  Due to incorrect locking in task.c, it was possible
for task.c to heap-use-after-free on the task-log message causing
corruption or crashing during import of ldif.

Fix Description:  Correct the locking to be after the mods are
commited to prevent the realloc from freeing before mod completes.

https://pagure.io/389-ds-base/issue/49444

Author: wibrown

Review by: lkrispen (Thank you!)

- - - - -
938dfb77 by Simon Pichugin at 2017-11-24T11:46:54Z
Issue 47536 - Add Python 3 support and move test case to suites

Description: Fix apply_mods function. Fix Python 3 issues.
Move the test case to sasl/regression_test.py.

https://pagure.io/389-ds-base/issue/47536

Reviewed by: wibrown (Thanks!)

- - - - -
8a7bee66 by William Brown at 2017-11-28T15:58:47Z
Ticket 49461 - Improve db2index handling for test 49290

Bug Description:  db2index has limited parameters we can accept,
including the types of attributes we can index. Specificatlly,
we must take an index name, else we do not re-index (look at
ns-slapd, we require -t or -T, and if we do, we require them
to have a value.

Fix Description:  Fix lib389 to match the assertions of ns-slapd
with regard to db2index behaviour.

https://pagure.io/389-ds-base/issue/49461

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
92209bf9 by William Brown at 2017-11-28T16:12:58Z
Ticket 3 lib389 - python 3 support for subset of pwd cases

Bug Description:  python 3 support for pwd test cases.

Fix Description:  Add support for python 3 to pwd test cases. Add
the replace_many function to improve our cases cleanliness
over apply_mods

https://pagure.io/lib389/issue/3

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
ead3168d by William Brown at 2017-11-29T07:59:08Z
Ticket 49475 - docker poc improvements.

Bug Description:  Improve the docker poc to support intree builds.
Add a .mk file to help mak ethis easier for people to use.

Fix Description:  Improve our specfile dependencies, our makefile
to use a with perl, and dockerfile to build from intree.

https://pagure.io/389-ds-base/issue/49475

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
26f78c1b by William Brown at 2017-11-29T09:04:40Z
Ticket 49325 - fix rust linking.

Bug Description:  An issue with the rpm was found that the way
the static .a was built would cause a missing library error when
the rpm was distributed.

Fix Description:  Because both cargo and automake are opinionated
and stubborn, this leaves the option as have cargo generate the .so
but write manual rules for install and linking. This has been tested
now with non-prefix, prefix, rpm build, copr.

https://pagure.io/389-ds-base/issue/49325

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
af11fb88 by Simon Pichugin at 2017-11-30T09:48:41Z
Issue 49374 - Add CI test case

Description: Add a test case to suites/config/regression_test.py.
Test that we can start the server after editing dse.ldif file
where we set nsslapd-errorlog-maxlogsize and nsslapd-errorlog-logmaxdiskspace
in a different order.

https://pagure.io/389-ds-base/issue/49374

Reviewed by: wibrown (Thanks!)

- - - - -
4f2207a3 by William Brown at 2017-12-01T10:44:09Z
Ticket 49449 - Load sysctl values on rpm upgrade.

Bug Description:  We provide sysctl values. We should apply them
on upgrade or installation to prevent a need to reboot.

Fix Description:  Add a post install script to load the sysctl
values from the system as they wolud be on a reboot. Note this
may not work in all cases (ie mock, container etc) so we have
to mask failures "just in case".

https://pagure.io/389-ds-base/issue/49449

Author: wibrown

Review by: tbordaz (Thank you!)

- - - - -
f75cfbce by William Brown at 2017-12-01T10:51:02Z
Ticket 49474 - sasl allow mechs does not operate correctly

Bug Description:  In a fix to sasl allowed mechs, the logic
was not properly configured.

Fix Description:  Alter the ids_sasl_supported_mech to be
clearer and simpler in it's design.

https://pagure.io/389-ds-base/issue/49474

Author: wibrown

Review by: tbordaz (Thank you!)

- - - - -
5530af50 by William Brown at 2017-12-04T15:39:24Z
Ticket 49480 - Improvements to support IPA install.

Bug Description:  The current state o fthe lib389 project needs
some tweaks to support IPA installs calling it as an API.

Fix Description:  Improve the allocate interface to make it
easier to setup instances.

Improve our objects to have an ipa root domain.

Enable ldapi socket path by default in install as IPA
expcets it.

https://pagure.io/389-ds-base/issue/49480

Author: wibrown

Review by: spichugi (thanks)

- - - - -
433f1149 by alisha17 at 2017-12-04T15:56:09Z
Ticket 49479 - Remove unused 'batch' argument from lib389

Bug Description: Batch is unused in lib389.

Fix Description:  Remove batch from lib389.

https://pagure.io/389-ds-base/issue/49479

Author: Alisha Aneja

Review by: wibrown, spichugi (Thanks Alisha!)

- - - - -
71d72285 by Viktor Ashirov at 2017-12-05T14:11:34Z
Issue 49485 - Typo in gccsec_defs

Bug Description:
In configure.ac gcc parameters for --enable-gcc-security have a typo:
-D_FORITY_SOURCE=2 instead of -D_FORTIFY_SOURCE=2

https://pagure.io/389-ds-base/issue/49485

Reviewed by: wibrown (Thanks!)

- - - - -
e741e40f by William Brown at 2017-12-06T10:51:49Z
Ticket 49445 - Memory leak in ldif2db

Bug Description:  Memory leak in ldif2db of the instance
name selected for the import

Fix Description:  Free the affected memory.

https://pagure.io/389-ds-base/issue/49445

Author: wibrown

Review by: lkrispen (thanks)

- - - - -
40ab212f by William Brown at 2017-12-06T10:51:56Z
Ticket 49445 - Improve regression test to detect memory leak.

Bug Description:  Memory leak in ldif2db of the instance
name selected for the import

Fix Description:  Update the test case to work on python 3,
which involves changes to tot_init test, and improvements
to dbgen to create the correct objects.

https://pagure.io/389-ds-base/issue/49445

Author: wibrown

Review by: lkrispen, spichugi (Thanks!)

- - - - -
876e326a by William Brown at 2017-12-06T10:52:05Z
Ticket 49486 - change ns stress core to use absolute int width.

Bug Description:  On some platforms the int width detection does
not correctly function and may cause inconsistent results with
__atomic family instructions.

Fix Description:  Convert the un-sized atomic calls in ns
stress core to _8 byte width.

https://pagure.io/389-ds-base/issue/49486

Author: wibrown

Review by: vashirov, lkrispen (Thanks!)

- - - - -
72bff95e by William Brown at 2017-12-06T10:52:14Z
Ticket 49484 - Minor cli tool fixes.

Bug Description:  Some commands stopped working.

Fix Description:  Minor fixes to backend class, initialise
and setup to work in more cases correctly.

https://pagure.io/389-ds-base/issue/49484

Author: wibrown

Review by: spichugi (Thanks)

- - - - -
b66b7126 by Amita Sharma at 2017-12-07T12:22:46Z
Issue 49443 - Add CI test case

Description: Add a test case to suites/filter/filter_test.py.
Test that ldapsearch with scope one returns only one entry.

https://pagure.io/389-ds-base/issue/49443

Reviewed by: wibrown (Thanks!) spichugi

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
20efeea7 by William Brown at 2017-12-07T16:30:47Z
Ticket 49470 - overflow in pblock_get

Bug Description:  While getting the connection id we used an int
not a uint64_t

Fix Description:  Make the stack size uint64_t instead.

https://pagure.io/389-ds-base/issue/49470

Author: wibrown

Review by: tbordaz (thanks)

- - - - -
d9ad6fd9 by William Brown at 2017-12-08T10:19:43Z
Ticket 49218 - Certmap - support TLS tests

Bug Description:  This adds support for pluggable certificate
mapping libraries. To achieve this, this replaces the existing
baked in certificate mapping code.

Fix Description: Improve our tls tests to cover more cases,
support external signing cas, user certs, and addition of
TLS by default to tests. This fixes some tests to use the
new interfaces, as well as extending topologies to allow tls
enabling.

https://pagure.io/389-ds-base/issue/49218
https://pagure.io/lib389/issue/95
https://pagure.io/lib389/issue/84

Author: wibrown

Review by: spichugi (Thank you!)

- - - - -
1418fc32 by William Brown at 2017-12-08T10:22:14Z
Ticket 48184 - close connections at shutdown cleanly.

Bug Description:  During shutdown we would not close connections.
In the past this may have just been an annoyance, but now with the way
nunc-stans works, io events can still trigger on open xeisting connectinos
during shutdown.

Fix Description:  Close connections during shutdown rather than
leaving them alive.

https://pagure.io/389-ds-base/issue/48184

Author: wibrown

Review by: lkrispen, vashirov (Thank you!)

- - - - -
5991388c by Thierry Bordaz at 2017-12-08T10:42:12Z
Ticket 49471 - heap-buffer-overflow in ss_unescape

Bug Description:
	Two problems here
		- when searching for wildcard and escape char, ss_unescape assumes the string
		  is at least 3 chars longs. So memcmp can overflow a shorter string
		- while splitting a string into substring pattern, it loops over
		  wildcard and can overpass the string end

Fix Description:
	For the first problem, it checks the string size is long enough to memcmp
        a wildcard or an escape
	For the second it exits from the loop  as soon as the end of the string is reached

https://pagure.io/389-ds-base/issue/49471

Reviewed by: William Brown

Platforms tested: F23

Flag Day: no

Doc impact: no

- - - - -
b5e840b6 by William Brown at 2017-12-11T09:28:02Z
Ticket 49494 - python 2 bytes mode.

Bug Description:  a piece of python 2 support was removed that
affected some tests.

Fix Description:  Add back the suport for non-bytes mode.

https://pagure.io/389-ds-base/issue/49494

Author: wibrown

Review by: tbordaz (Thanks)

- - - - -
c0346d5f by William Brown at 2017-12-11T16:16:01Z
Ticket 49495 - Fix memory management is vattr.

Bug Description:  During the fix for
https://pagure.io/389-ds-base/issue/49436 a issue was exposed
in how registration of attributes to cos work. With the change
to handle -> attr link, this exposed that cos treats each attribute
and template pair as a new type for the handle. As  aresult, this
caused the sp_list to create a long linked list of M*N entries
for each attr - template value. Obviously, this is extremely
slow to traverse during a search!

Fix Description:  Undo part of the SLL next change and convert
to reference counting. The issue remains that there is a defect
in how cos handles attribute registration, but this can not be
resolved without a significant rearchitecture of the code
related to virtual attributes.

https://pagure.io/389-ds-base/issue/49495

Author: wibrown

Review by: tbordaz, lkrispen (Thanks!)

- - - - -
ab61eff1 by William Brown at 2017-12-11T16:37:15Z
Ticket 49495 - cos stress test and improvements.

Bug Description:  We previously had no way to test the cos plugin.

Fix Description:  Add cos types, and a stress test for the
template system to demonstrate the issue with 49495

https://pagure.io/389-ds-base/issue/49495

Author: wibrown

Review by: spichugi (Thanks!)

- - - - -
ca855b89 by Amita Sharma at 2017-12-13T17:00:59Z
Ticket 49431 - Add CI test case

Description: Add a test case to suites/replication/regression_test.py
Test that replicated MODRDN does not break replication

https://fedorahosted.org/389/ticket/49431

Reviewed by: spichugi, lkrispen, wibrown (thanks!)

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
5f3e4be2 by Akshay Adhikari at 2017-12-14T13:00:23Z
Issue 49379 - Add Python 3 support to CI test

Bug Description: lib389/rootdse.py library did not support python 3

Fix Description: use byte type for the supportedSASLMechanisms

https://pagure.io/389-ds-base/issue/49379

Reviewed by: Simon Pichugin

- - - - -
c68eaed8 by Ludwig Krispenz at 2017-12-14T14:37:27Z
Ticket 49493 - heap use after free in csn_as_string

Bug: If write_changlog_and_ruv failed teh csn pending list was not properly
     cleand and references to the prim csn were kept, but the prim csn was reset

Fix: check the return code for the mmr postop plugin and aset error codes properly
     that will triger cancel_opcsn

Reviewed by: Thierry, thanks
Tested by: Viktor, thanks

- - - - -
debe2781 by alisha17 at 2018-01-02T15:39:57Z
Issue #35 - dsconf automember support

Bug Description: Add support for managing automember to dsconf

Fix Description: Initial patch which adds AutoMembershipPlugin, AutoMembershipDefinition
and AutoMembershipDefinitions classes to plugins.py and adds tests for checking valid scope,
valid filter and if user is correctly added to the group.

https://pagure.io/lib389/issue/35

Author: Alisha Aneja

Review by: ???

- - - - -
87609c0a by William Brown at 2018-01-04T00:25:53Z
Ticket 49474 - Improve GSSAPI testing capability

Bug Description:  GSSAPI is difficult to test correctly. Having
support for it as a topology in lib389 will mak ethis easier.

Note that these tests require specific dns and hosts settings.

Fix Description:  Improve the ability to integrate accounts with
principles, bind them with GSSAPI, oru ability to return valid
uris with krb realms, our saslmap capabilities and create a
single master gssapi topology.

https://pagure.io/389-ds-base/issue/49474

Author: wibrown

Review by: spichugi (Thank you!)

- - - - -
d1ef7077 by William Brown at 2018-01-04T05:31:48Z
Ticket 49515 - cannot link, missing -fPIC

Bug Description:  When building the server with --disable-gcc-security,
  due to an issue in autotools linker flag emission on RPM based
  system, -fPIC is missing. --enable-gcc-security masks this by
  providing -specs that add -fPIC when required on the types.

  This causes the build to fail in some conditions,

Fix Description:  Detect and add the rpm cc specs if present
  on all builds.

https://pagure.io/389-ds-base/issue/49515

Author: wibrown

Review by: mreynolds (Thanks!)

- - - - -
040a6270 by Mark Reynolds at 2018-01-04T15:32:07Z
Ticket 49512 - Add initial Cockpit UI Plugin

Description:  Get the inital framwork intothe repo so everyone can
              work on it

https://pagure.io/389-ds-base/issue/49512

Reviewed by: wibrown(Thanks!)

- - - - -
6603fb73 by Mark Reynolds at 2018-01-04T16:12:20Z
Ticket 49512 - remove backup directories from cockpit source

Description:  Remove accidentally added backups dir

- - - - -
6647fba1 by Akshay Adhikari at 2018-01-05T12:14:22Z
Issue 49312 - Added a new test case for "-D configdir"

Bug Description: "pwdhash -D configdir" uses the DS default hashing algorithm.

Fix Description: Add a test case that tests the change of default pwdhash scheme.

https://pagure.io/389-ds-base/issue/49312

Reviewed by: wibrown, spichugi

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
bf59861b by Thierry Bordaz at 2018-01-05T15:04:42Z
Ticket 49523 - memberof: schema violation error message is confusing as memberof will likely repair target entry

Bug Description:
	When memberof is enabled it adds 'memberof' attribute to members entries.
	If a member entry has not the appropriate objectclass to support 'memberof' attribute an ERR is logged.

	ERR - oc_check_allowed_sv - Entry "cn=user_1,ou=People,dc=example,dc=com" -- attribute "memberOf" not allowed

	This is confusing because memberof will catch this violation and may try to repair it.
	So although this message is alarming, the target entry may finally have the 'memberof' attribute.

	This is especially confusing since https://pagure.io/389-ds-base/issue/48985 where the repair operation
	is done by default (if schema is violated)

	We can not (and should not) eliminate the schema violation message.
	But memberof should log a additional warning (beside the schema violation msg) stating it repaired the violation.

Fix Description:

	Add a warning message upon repair operation
		ERR - oc_check_allowed_sv - Entry "<entry_dn>" -- attribute "memberOf" not allowed
		WARN - memberof-plugin - Entry <entry_dn> - schema violation caught - repair operation succeeded

https://pagure.io/389-ds-base/issue/49523

Reviewed by: Mark Reynolds

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
9e23b4f8 by Mark Reynolds at 2018-01-08T01:28:18Z
Ticket 49517 - Cockput UI - revise config layout

Description:  This patch starts to revise the config layout

https://pagure.io/389-ds-base/issue/49517

Reviewed by: wibrown & tbordaz (Thanks!)

- - - - -
d79102f6 by Mark Reynolds at 2018-01-08T01:35:05Z
Ticket 49517 - Cockpit UI - Add correct png files

Description: Add the correct png files

https://pagure.io/389-ds-base/issue/49517

- - - - -
790be09f by Mark Reynolds at 2018-01-08T16:34:02Z
Ticket 49524 - Password policy: minimum token length fails
 when the token length is equal to attribute length

Bug Description:  The token checking breaks when the password is the
                  exact value of the entry attribute.

Fix Description:  Remove the "equal" part of the string comparisons.

https://pagure.io/389-ds-base/issue/49524

Reviewed by: firstyear & spichugi(Thanks!!)

- - - - -
f102eab6 by Simon Pichugin at 2018-01-09T09:39:55Z
Ticket 49523 - Refactor CI test

Description: Add Python 3 support, move the test case
to suites/memberof_plugin/regression-test.py and
refactor its structure with lib389 wrappers.

https://pagure.io/389-ds-base/issue/49523

Reviewed by: mreynolds (Thanks!)

- - - - -
2be0fad1 by Mark Reynolds at 2018-01-09T15:20:12Z
Ticket 49512 - Add ds-cockpit-setup to rpm spec file

Description:  Was missing the script in the rpmspec file

https://pagure.io/389-ds-base/issue/49512

Reviewed by: mreynolds (one line commit rule)

- - - - -
44c2d8f5 by Mark Reynolds at 2018-01-09T21:13:51Z
Ticket 49520 - Cockpit UI - Add database chaining HTML

Description:  Add chaining pages/forms

https://pagure.io/389-ds-base/issue/49520

Reviewed by: ?

- - - - -
00d04b07 by Amita Sharma at 2018-01-10T11:16:27Z
Ticket 48118 - Add CI test case

Description: Add a test case to suites/memberof_plugin/regression_test.py
Test that replication does not break, after DS stop-start, due to changelog reset,
while using member_of plugin

https://pagure.io/389-ds-base/issue/48118

Reviewed by: lkrispen, spichugi

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
3aa02b69 by William Brown at 2018-01-11T00:48:08Z
Ticket 49508 - memory leak in cn=replica plugin setup

Bug Description:  during db2ldif a memory leak in cn=replica
mt node setup was found.

Fix Description:  Free the memory!

https://pagure.io/389-ds-base/issue/49508

Author: wibrown

Review by: mreynolds (Thank you!)

- - - - -
fa71a0a9 by Ludwig Krispenz at 2018-01-11T14:17:56Z
   Ticket 49278 - GetEffectiveRights gives false-negative

    Bug: If geteffective rights was issued for an non existing entry the
         mechanism to genrate a template entry no longer worked and no results were
         returned.
    Fix: Improve the handling in itreating the result set, so that template entries (if
         requested) are genereated and are not applied to existing entries.
         Also some code cleanup in iterate()
    Reviewed by: Thierry, thanks

- - - - -
339caec2 by Ludwig Krispenz at 2018-01-11T15:47:05Z
Ticket 49446 - cleanallruv should ignore cleaned replica Id in processing changelog if in force mode

Bug: If the startcsn is calculated based on a cleaned rid, it could be missing from the changelog.

Fix: In force mode we do not care that the topology gets in sync for the cleaned RID, so we can ignore it
     in an earlier stage, instead of setting it to precleane only.

Reviewed by: Thierry, thanks

- - - - -
575d9e29 by Ludwig Krispenz at 2018-01-11T15:54:54Z
Ticket 49413 - Changelog trimming ignores disabled replica-agreement

Bug: if a replication agreement is disabled it is not taken into account when
     changelog trimming determines where to stop.
     If the agreement is reenabled later replication can fail

Fix: do not ignore disabled agreements in changelog trimming

Reviewed by: Thierry, thanks

- - - - -
0457ea62 by William Brown at 2018-01-12T00:00:15Z
Ticket 49474 - purge saslmaps before gssapi test

Bug Description:  In the GSSAPI test, depending on enviroment
some of the default maps can "work" or they "may not". This is
due to inconsistency in the behaviour of the cyrus-sasl
library and it's assignment of realm into requests.

Fix Description:  Purge all maps by default, and just use the
provided test sasl map - this is a single "source of truth"
and thus allows us to test assertions easier.

https://pagure.io/389-ds-base/issue/49474

Author: wibrown

Review by: spichugi (Thank you!)

- - - - -
d3ba2287 by William Brown at 2018-01-12T00:02:04Z
Ticket 49527 - Improve ds* cli tool testing

Bug Description:  As we get closer to release it's important
our tests work correctly and are comprehensive.

Fix Description:  Improve the directory manager test, backend test
user test and correct some incompatability with memberof test.

https://pagure.io/389-ds-base/issue/49527

Author: wibrown

Review by: mreynolds, spichugi (Thanks!)

- - - - -
3a161b23 by Thierry Bordaz at 2018-01-12T09:50:59Z
Ticket 49509 - Indexing of internationalized matching rules is failing

Bug Description:
    Indexing of the internationalized matching rules tests if a
    matching rule indexer handle or not a given OID.
    A side effect of https://pagure.io/389-ds-base/issue/49097 is that
    the returned indexing callbacks are lost.
    Indeed, the indexing callbacks (and potentially others fields) were
    stored in the temporary pblock that was memcpy to the provided
    pblock in case of success

Fix Description:
    The fix basically restores the previous behavior but do not
    memcpy pblock. It read/store the pblock fields that are
    inputs/outputs of slapi_mr_indexer_create.

https://pagure.io/389-ds-base/issue/49509

Reviewed by: Ludwig Krispenz

Platforms tested: F23

Flag Day: no

Doc impact: no

- - - - -
7e27face by Mark Reynolds at 2018-01-12T13:39:03Z
Ticket 49529 - Fix Coverity warnings: invalid deferences

Description:  So many of the warnings were false positives, but
              I "fixed" 90% of them anyway for these two reasons:

              One, it's possible that a future change could actually
              result in a NULL pointer being referenced.

              Two, it would be nice to stop these coverity warnings
              so we can focus on real warnings.  Auto waivers also
              don't always work as the surrounding code changes.

https://pagure.io/389-ds-base/issue/49529

Reviewed by: firstyear (Thanks!)

- - - - -
84483697 by Thierry Bordaz at 2018-01-12T14:20:38Z
Ticket 49463 - After cleanALLruv, there is a flow of keep alive DEL

Bug Description:
	When cleanAllRuv is launched, it spawn cleanAllRuv on all replicas.
	Each replica will clean its changelog and database RUV AND in addition
	will DEL the keep alive entry of the target ReplicaID.
	So for the same entry (keep alive) there will be as many DEL as there are replicas

	This flow of DEL is useless as only one DEL is enough.
	In addition because of https://pagure.io/389-ds-base/issue/49466, replication may
	loop on each of those DELs.

Fix Description:
	The fix is only to prevent the flow of DEL.
	It adds a flag ('original_task') in the task payload.
	The server receiving the task (replica_execute_cleanall_ruv_task) flags the
	task as 'original_task'.
	In the opposite, the propagated cleanAllRuv (multimaster_extop_cleanruv) does
	not flag the task as 'original_task'
	Only original task does the DEL of the keep alive entry.
	Note the propageted payload (extop) is not changed. In a mixed version
	environment "old" servers will DEL the keep alive and flow can still happen

https://pagure.io/389-ds-base/issue/49466

Reviewed by: Ludwig Krispenz

Platforms tested: F23

Flag Day: no

Doc impact: no

- - - - -
700d7422 by Mark Reynolds at 2018-01-12T15:40:48Z
Ticket 49531 - coverity issues - fix memory leaks

Description:  There were two false positives around pwpolicy struct
              being leaked, but it is freed when the pblock is
              destroyed.  The other two leaks were real, but they
              only occurred during error conditions.

https://pagure.io/389-ds-base/issue/49531

Reviewed by: lkrispen (Thanks!)

- - - - -
05907ae0 by Mark Reynolds at 2018-01-12T16:56:54Z
Ticket 49532 - coverity issues - fix compiler warnings & clang issues

Description:  Fixed all the warnings

https://pagure.io/389-ds-base/issue/49532

Reviewed by: tbordaz(Thanks!)

- - - - -
7658232c by Mark Reynolds at 2018-01-16T15:14:34Z
Ticket 49534 - Fix coverity issues and regression

Description:  Fix regression introdcued in the previous coverity patch.

              Also fixed many other coverity issues.

https://pagure.io/389-ds-base/issue/49534

Reviewed by:  wibrown, tbordaz, lkrispen(Thanks!)

- - - - -
e163c443 by William Brown at 2018-01-17T13:11:01Z
Ticket 49516 - Add python 3 support for replication suite

Bug Description:  Add support for python 3 to the replication
suite. Additionally, update many types to correct be dsldapobject
designed and created.

Fix Description:

 * Update all repl test to use python 3
 * Mark tests that cannot be currently executed due to server
   limitations
 * Add replication manager types to allow proper coordination
   of replicas.
 * Improve supprot for tls with replication tests.

https://pagure.io/389-ds-base/issue/49516

Author: wibrown

Review by: spichugi

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
66ee27de by Mark Reynolds at 2018-01-17T19:31:17Z
Ticket 49526 - Improve create_test.py script

Description:  Autofill "setup" docstring, and fix python warning about
              passing a string to "main"

https://pagure.io/389-ds-base/issue/49526

Reviewed by:  spichugi (Thanks!)

- - - - -
7b03c31d by William Brown at 2018-01-18T02:35:12Z
Ticket 49537 - allow asan to build with stable rustc

Bug Description:  Rustc has a policy of only allowing -Z flags on
  nightly - but there is an environment variable that allows
  unblocking these on stable releases.

Fix Description:  Add RUSTC_BOOTSTRAP=1 to Makefile.am

https://pagure.io/389-ds-base/issue/49537

Author: wibrown

Review by: mreynolds (Thank you!)

- - - - -
b086d416 by William Brown at 2018-01-18T04:01:47Z
Ticket 49425 - improve demo objects for install

Bug Description:  Improve demo objects for install

Fix Description:  Change the tree a tiny bit - add hidden
389 container, add ous, use better aci examples. This also
adds a set of tests to assert the default aci's work as
advertised.

https://pagure.io/389-ds-base/issue/49425

Author: wibrown

Review by: mreynolds, spichugi, tbordaz, lkrispen (Thank you all!)

- - - - -
c8b388bf by Mark Reynolds at 2018-01-18T14:58:17Z
Ticket 49370 - Add all the password policy defaults to a new
 local policy

Bug Description:  When processing a local password policy we were not pulling
                  in the defaults for the "on/off" settings.  This patch
                  addresses that.

Fix Description:  Create common default init functions for all password policies

https://pagure.io/389-ds-base/issue/49370

Reviewed by: tbordaz, wibrown, and spichugi (Thanks!!!)

- - - - -
ebb00a41 by Mark Reynolds at 2018-01-18T18:17:08Z
Ticket 49541 - repl config should not allow rid 65535 for masters

Description:  Reject adding a replica config entry with a rid of 65535 or higher,
              and prevent setting master's rid to 65535 or higher.

https://pagure.io/389-ds-base/issue/49541

Reviewed by: mreynolds(one line commit rule)

- - - - -
f59c39e1 by Viktor Ashirov at 2018-01-19T08:18:28Z
Issue 49542 - Unpackaged files on el7 break rpm build

Bug Description:
Some files are not created during build on el7, but are mentioned in
%files section.

Fix Description:
Package them on only on Fedora and EL>7

https://pagure.io/389-ds-base/issue/49542

Reviewed by: wibrown (Thanks!)

- - - - -
af8cda08 by William Brown at 2018-01-23T01:24:10Z
Ticket 49544 - cli release preperation, group improvements

Bug Description:  Improvements to the cli getting ready for user testing.
  The group handling did not have tests nor the ability to add or remove
  members.

Fix Description:
 This adds support for adding and removing users to groups, as well as
 testing that. To improve the tests, duplicate code from topology was
 removed in the cli side, and placed into topologies.py

https://pagure.io/389-ds-base/issue/49544

Author: wibrown

Review by: mreynolds, spichugi (Thanks!)

- - - - -
b3768e60 by Mark Reynolds at 2018-01-24T19:24:08Z
Ticket 49534 - Fix coverity regression

Description:  In automembers plugin a free was in the wrong spot
              which later led to a double free for the "rule".

https://pagure.io/389-ds-base/issue/49534

Reviewed by: mreynolds (one line commit rule)

- - - - -
cb7bda3e by Thierry Bordaz at 2018-01-26T15:42:46Z
Ticket 49540 - Indexing task is reported finished too early regarding the backend status

Bug Description:
	If a task complete successfully, its status is updated before the backend
	can receive update.

Fix Description:
	postpone the task status update after backend is reenabled

https://pagure.io/389-ds-base/issue/49540

Reviewed by: Ludwig Krispenz

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
f95b8e7f by Mark Reynolds at 2018-01-29T14:37:49Z
Ticket 49552 - Fix build issues on F28

Description:  The newer version of gcc on F28 is stricter with linking.  Also,
              unistd.h no longer includes the "unix crypt", so we need to revert
              to crypt.h

              Added these LDFLAGS to the spec file: -Wl,-z,defs

https://pagure.io/389-ds-base/issue/49552

Reviewed by: firstyear & lslebodn (Thanks!!)

- - - - -
d86e0f96 by Mark Reynolds at 2018-01-29T17:44:10Z
Ticket 49370 - Crash when using a global and local pw
 policies

Description:  This a regression from the previous patch.  We were
              accidently using a reference to the global pw policy
              password storage scheme, which was getting freed after
              pblock was done from an operation.  The next operation
              then used(and double freed) this memory on the next
              operation.

https://pagure.io/389-ds-base/issue/49370

Reviewed by: tbordaz (Thanks!)

- - - - -
56f98815 by Mark Reynolds at 2018-01-29T23:39:11Z
Bump version to 1.4.0.4

- - - - -
8ce1d38b by Mark Reynolds at 2018-01-30T04:53:59Z
Ticket 49548 - Cockpit UI - installer should also setup
 Cockpit

Description:  Add option to installer to also configure Cockpit:
               - Add port to firewall (if running),
               - Enable/start the Cockpit socket.

https://pagure.io/389-ds-base/issue/49548

Reviewed by: firstyear(Thanks!)

- - - - -
e435e96b by William Brown at 2018-01-30T05:46:33Z
Ticket 49544 - Double check pw prompts

Bug Description:  We did not prompt for password change twice.

Fix Description:  Prompt twice when requested to ensure we don't
mess up inputs.

https://pagure.io/389-ds-base/issue/49544

Author: wibrown

Review by: spicugi (Thanks!)

- - - - -
003fa4d0 by Amita Sharma at 2018-01-30T07:57:13Z
Ticket 49523 - Fixed skipif marker, topology fixture and log message

Description: Fixed test_scheme_violation_errors_logged test case
for skipif marker to have correct build number, fixed topology fixture
and fixed log.info for memberof attribute.

https://fedorahosted.org/389/ticket/49523

Reviewed by: Simon Pichugin (thanks)

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
b6c1221e by Viktor Ashirov at 2018-01-30T13:08:37Z
Issue 49530 - Add pseudolocalization option for dbgen

Bug Description:
Pseudolocalization can be used to test matching rules
and indexing of UTF-8 strings in various attributes.

Fix Description:
Add new function 'pseudlocalize' to utils module of lib389 that accepts
string and returns pseudolocalized string. For example:
"389 Directory Server" will become "③⑧⑨ Ðîŕéçţöŕý Šéŕṽéŕ"

Add new optional argument 'pseudol10n' for dbgen module that is False by
default. If True, attributes like cn, sn, title, description, etc will
be pseudolocalized.

https://pagure.io/389-ds-base/issue/49530

Reviewed by: lkrispen, tbordaz, spichugi (Thank you all!)

- - - - -
8743426c by Mark Reynolds at 2018-01-30T17:13:25Z
Ticket 49400 - Make CLANG configurable

Description:  Make clang configurable and off by default

https://pagure.io/389-ds-base/issue/49400

Reviewed by: viktor(Thanks!)

- - - - -
677a502c by William Brown at 2018-01-31T02:31:26Z
Ticket 49554 - update readme

Bug Description:  Our readme needs improving.

Fix Description:  Improve it with build steps and links, change
to md format.

https://pagure.io/389-ds-base/issue/49554

Author: wibrown

Review by: mreynolds, vashirov (Thanks!)

- - - - -
6aa2acdc by Ludwig Krispenz at 2018-01-31T12:42:12Z
CVE-2017-15134 389-ds-base: Remote DoS via search filters in
 slapi_filter_sprintf

Description: Improper handling of a search filter in slapi_filter_sprintf
             in slapd/util.c can lead to remote server crash and denial
             of service.

- - - - -
ee62d3d5 by Mark Reynolds at 2018-01-31T13:22:00Z
Ticket 49546 - Fix broken snmp MIB file

Description:  Converted our MIB file to be SMIv2 compliant.  Also
              slightly rearranged the entities to be more organized.

https://pagure.io/389-ds-base/issue/49546

Reviewed by: firstyear & nkinder (Thanks!!)

- - - - -
d1ca0fec by Mark Reynolds at 2018-01-31T19:27:51Z
Ticket 49554 - Update Makefile for README.md

Description:  The Makefile needed to be updated for README.md

https://pagure.io/389-ds-base/issue/49554

Reviewed by: mreynolds(one line commit rule)

- - - - -
02ca11bf by Mark Reynolds at 2018-01-31T19:35:16Z
Bump version to 1.4.0.5

- - - - -
0f95ada9 by Akshay Adhikari at 2018-02-01T11:45:31Z
Issue 48006 - Add a new CI test case

Bug Description: Missing warning for invalid replica backoff configuration

Fix Description: Add a test case that will check the invalid replica backoff configuration
(the case when nsds5ReplicaBackoffMin is set to the bigger value than nsds5ReplicaBackoffMax)

https://pagure.io/389-ds-base/issue/48006

Reviewed by: spichugi, wibrown (Thanks!)

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
45463c7c by William Brown at 2018-02-02T05:35:35Z
Ticket 49544 - cli release preperation

Bug Description:  Improvements to the cli getting ready for user testing.
 Fixes for output rendering (verbose vs normal), automatic help display,
 raising assertion messages properly during setup when constraints are
 violated, capturing output correctly from subcommands.

Fix Description:  Improve the logging handler by wrapping a custom
setup function that can purge and reset handlers.

Add automatic help display if not action selected.

Add assert_c for compiled asserts. python removes assert in optimised
builds, and we need custom messages for the frontend anyway.

Some commands would display to stderr, capture that.

https://pagure.io/389-ds-base/issue/49544

Author: wibrown

Review by: spichugi (thanks)

- - - - -
0c8906b0 by alisha17 at 2018-02-02T05:50:18Z
Bug Description: Add support for managing automember to dsconf

Fix Description: Added the dsconf support and tests for creating, removing, editing and showing
the automember definitions.

https://pagure.io/lib389/issue/35

Author: Alisha Aneja

Review by: wibrown (thank you so much!!!)

- - - - -
66ecdf99 by Amita Sharma at 2018-02-06T15:54:12Z
Ticket 49446 - Add CI test case

Description: Add a test case to suites/replication/regression_test.py
Test that cleanallruv could not break replication if startCsn originated
from deleted replica

https://pagure.io/389-ds-base/issue/49446

Reviewed by: Simon, Thierry and William

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
ca8f1fd0 by Mark Reynolds at 2018-02-06T18:01:46Z
Ticket 49557 - Add config option for checking CRL on outbound SSL Connections

Bug Description:  There are cases where a CRL is not available during an outbound
                  replication connection.  This is seen as an error by openldap,
                  and the connection fails.

Fix Description:  Add on/off option for checking the CRL.  The default is not to
                  check the CRL.

https://pagure.io/389-ds-base/issue/49557

Reviewed by: wibrown, Ludwig Krispenz, Thierry Bordaz

- - - - -
8304caec by Thierry Bordaz at 2018-02-06T23:36:09Z
Ticket 49560 - nsslapd-extract-pemfiles should be enabled by default as openldap is moving to openssl

Bug Description:
	Due to a change in the OpenLDAP client libraries (switching from NSS to OpenSSL),
	the TLS options LDAP_OPT_X_TLS_CACERTFILE, LDAP_OPT_X_TLS_KEYFILE, LDAP_OPT_X_TLS_CERTFILE,
	need to specify path to PEM files.

	Those PEM files are extracted from the key/certs from the NSS db in /etc/dirsrv/slapd-xxx

	Those files are extracted if the option (under 'cn=config') nsslapd-extract-pemfiles is set to 'on'.

	The default value is 'off', that prevent secure outgoing connection.

Fix Description:

	Enable nsslapd-extract-pemfiles by default
	Then when establishing an outgoing connection, if it is not using NSS crypto layer
	and the pem files have been extracted then use the PEM files

https://pagure.io/389-ds-base/issue/49560

Reviewed by: mreynolds

Platforms tested: RHEL 7.5

Flag Day: no

Doc impact: no

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
a65e6da6 by Akshay Adhikari at 2018-02-08T10:36:31Z
Issue 49278 - Add a new CI test case

Bug Description: GetEffectiveRights gives false-negative with ACIs containing targetfilter

Fix Description: Added tests cases that will check whether a user have entry-level-access
even where the access is conditional on a targetfilter.
An non-existing entry defined by objectclass has effective rights.

https://pagure.io/389-ds-base/issue/49278

Reviewed by:  lkrispen,spichugi

- - - - -
ad83e551 by Ludwig Krispenz at 2018-02-12T08:24:25Z
Ticket 49551 - v3 - correct handling of numsubordinates for cenotaphs and tombstone delete

        Bug: The ticket exposed several problems with tombstone handling.
                - tombstone entries of conflicts were not purged in tombstone purging
                - cenotaphs are tombstone, but the subordinate count was not managed properly
                - direct delete of tombstones failed with err=1
                - delete of entry with only conflict children failed correctly, but gave no hint why

        Fix: update the correct numsobordinates attribut for cenotaphs
             set proper flag in directly deleting a tombstone
             change search filter for tombstone purging to include ldapsubentries
             check for conflict children if a delete is rejected and add a message to the response

        Reviewed by; Thierry, William - thanks

- - - - -
d9706492 by Ludwig Krispenz at 2018-02-12T16:16:46Z
Ticket 49551 - fix memory leak found by coverity

- - - - -
7a54d9b3 by Ludwig Krispenz at 2018-02-13T13:42:44Z
fix compiler warning for const csn, caused by previous fix for t 49551

- - - - -
a3341c0e by Mark Reynolds at 2018-02-15T02:58:19Z
Ticket 49519 - Add more Cockpit UI content

Description:  Add recent updates to Cockpit UI

https://pagure.io/389-ds-base/issue/49519

Reviewed by: firstyear(Thanks!)

- - - - -
9e2009ae by Mark Reynolds at 2018-02-16T14:15:59Z
Ticket 49566 - ds-replcheck needs to work with hidden conflict entries

Description:  Conflict entries are now hidden and the tool needs to account
              for it.  The filter needs to include "objectclass=ldapsubentry"

              Added option to prompt for password, and cleaned up man page.

https://pagure.io/389-ds-base/issue/49566

Reviewed by: spichugi(Thanks!)

- - - - -
e62a4436 by Simon Pichugin at 2018-02-19T10:36:07Z
Issue 48085 - Add encryption cl5 test suite

Description: Add a test suite that checks AES and 3DES encryption algorithms.
Check unhashed#user#password for encryption in different circomstances.

The test suite based on Sankar Ramalingam test suite.

http://pagure.io/389-ds-base/issue/48085

Reviewed by: lkrispen, wibrown (Thanks!)

- - - - -
774ee3e4 by Mark Reynolds at 2018-02-19T16:06:06Z
Ticket 49568 - Fix integer overflow on 32bit platforms

Bug Description:  When generating the nsState information for the
                  uniqueid generator the timestamp was overflowing
                  on arm(and possibly other 32 bit platforms).  This
                  then broke the upgrade setup code when 50fixNsState.pl
                  was ran.  50fixNsState script also incorrectly converts
                  64bit nsState value to 32 bit

Fix Description:  Cast the current time (time_t) to the proper type when
                  when calculating the time since epoch.  Also do not
                  convert nsState during upgrade if it is already 64bit
                  value.

https://pagure.io/389-ds-base/issue/49568

Reviewed by: firstyear(Thanks!)

- - - - -
0d5214d0 by Mark Reynolds at 2018-02-20T13:54:41Z
Ticket 49296 - Fix race condition in connection code with
 anonymous limits

Bug Description:  When a connection first comes in we set the anonymous
                  resource limits (if set) before we do anything else.  The
                  way we check if the connection is "new" was flawed.  It
                  assumed the connection was new if no operations were
                  completed yet, but there was a small window between sending
                  the result and setting that the operation completed in the
                  connection struct.

                  So on a connection that binds and then does a search, when
                  the server sends the bind result the client sends the search,
                  but the search op/activity can be picked up before we set
                  c_opscompleted.  This opens a window where the code thinks
                  the search op is the first op(new connection), and it incorrectly
                  sets the anonymous limits for the bind dn.

Fix description:  Do not use c_opscompleted to determine if a connection is new,
                  instead use a new flag to set the connection "initialized",
                  which prevents the race condition.

https://pagure.io/389-ds-base/issue/49296

Reviewed by: firstyear(Thanks!)

- - - - -
14e413ae by Simon Pichugin at 2018-02-21T08:56:24Z
Issue 49043 - Add replica conflict test suite

Description: Add a test suite which checks replication conflict resolution
for basic operations like add, delete, modrdn, modify, operations on groups
with memberOf plugin enabled, managed entries operations and nested entries.
idm/user.py - add create_test_user method which allows default user creation
with given uid and guid.
_mapped_object.py - add delete_tree() method for DSLdaObject class
_constants.py - add access log level constants AccessLog(IntEnum) and ErrorLog(IntEnum)

Fix topologies tearDown part. Replace map() with list comprehensions, because
in Python 3, map() returns iterator and it doesnt execute the content on init.

https://pagure.io/389-ds-base/issue/49043

Reviewed by: wibrown, lkrispen (Thanks!)

- - - - -
ec4a669c by Ludwig Krispenz at 2018-02-22T14:54:04Z
ticket 49551 - correctly handle subordinates and tombstone numsubordinates

	Additional fix for a case where an ADD is directly turned into a tombstone and teh tombstone
	is resurrected as conflict

Reviewed by: Mark, thanks

- - - - -
c6d87264 by William Brown at 2018-02-27T00:20:16Z
Ticket 49447 -  PBKDF2 on upgrade

Bug Description:  We changed the default password hash mech to
PBKDF2 on upgrade - but if the upgrade from 1.3.x to 1.4.x,
this new plugin object doesn't exist. This causes the server to
fail to start.

Adittionally, one could imagine an admin error could cause this
also to be omitted

Fix Description:  Create and bootstrap the plugins on startup

https://pagure.io/389-ds-base/issue/49447

Author: wibrown

Review by: mreynolds

- - - - -
272628ba by William Brown at 2018-02-27T01:41:08Z
Add 'src/svrcore/' from commit 'bc12b6d56fcdd703b6a282ba57679a515484f54d'

git-subtree-dir: src/svrcore
git-subtree-mainline: ec4a669cd38bebe90abd3e0c9fb1d6ecd35ef153
git-subtree-split: bc12b6d56fcdd703b6a282ba57679a515484f54d

- - - - -
09f4003e by Ludwig Krispenz at 2018-02-27T12:56:14Z
Ticket 49161 - memberof fails if group is moved into scope

if the DEL part of the replace of memberof fails because it does not exist
just add the new memberof values

Reviwed by: Mark, thanks

- - - - -
389190a5 by William Brown at 2018-02-27T22:59:46Z
Ticket 49369 - merge svrcore into 389-ds-base

Bug Description:  In order to keep our project simple and clear
it's a good idea to keep our dependencies together. As the only
consumers and maintainers of svrcore, we shouldmake it part of
our project!

Fix Description:  Merge svrcore and it's history to
389-ds-base

https://pagure.io/389-ds-base/issue/49369

Author: wibrown

Review by: ???

- - - - -
c00ef655 by Simon Pichugin at 2018-03-01T11:46:54Z
Issue 49584 - Fix issues with paged_results test suite

Bug Description: The test fails because the instance now uses
hostname for ldap URL but the ACI is set to localhost URL.

Fix description: Get FQDN and IP addresses of the machine
and use them for ACI creation.

https://pagure.io/389-ds-base/issue/49584

Reviewed by: mreynolds, wibrown (Thanks!)

- - - - -
b831d6ac by Simon Pichugin at 2018-03-02T12:00:25Z
Issue 49572 - ns_job_wait race on condvar

Bug description: ns_job_persist_disarm_test were failing
because of a race condition.

Fix description: We need to lock the cb mutex before we arm
the job so the order of operations that happens there is
determined.
First we lock the mutex, then after the rearm it will
disarm it and will wait inside for the mutex unlock.
In a meanwhile, our testing thread will come to cond_wait
and it will unlock the mutex while waiting for the signal.
The mutex inside of callback will be acquired and cb_check will set to 1
and the signal will be sent. After that, cond_wait will be released and
the test will check for the right cb_check value.

https://pagure.io/389-ds-base/issue/49572

Reviewed by: wibrown (Thanks!)

- - - - -
14ce2fe0 by Thierry Bordaz at 2018-03-06T17:51:45Z
Ticket 49545 - final substring extended filter search returns invalid result

Bug Description:
	During a search (using extended filter with final substring), the server
	checks the filter before returning the matching entries.
	When checking the attribute value against the filter, it
	uses the wrong value.

Fix Description:
	Make suree it uses the right portion of the attribute value, in order
	to generate the keys to compare.

https://pagure.io/389-ds-base/issue/49545

Reviewed by: Ludwig Krispenz

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
ffbbae35 by Mark Reynolds at 2018-03-06T19:10:01Z
Bump version to 1.4.0.6

- - - - -
7d5c27ac by Simon Pichugin at 2018-03-07T08:26:42Z
Issue 49239 - Add a test suite for ds-replcheck tool RFE

Description: Check that ds-replcheck tool works with all types of connections
and it can show the disserence between instances (replica conflicts,
attribute inconsistencies, missing entries and number of tombstones)

https://pagure.io/389-ds-base/issue/49239

Reviewed by: mreynolds, wibrown (Thanks!)

- - - - -
73638d68 by Simon Pichugin at 2018-03-07T08:36:27Z
Issue 49560 - Add a test case for extract-pemfiles

Description: Add a test case to existing test suite:
replication/tls_client_auth_repl_test.py
It should test that replication works with 'on' and 'off'
values and the default vaue is right.

https://pagure.io/389-ds-base/issue/49560

Reviewed by: wibrown (Thanks!)

- - - - -
e6cc2780 by William Brown at 2018-03-07T22:21:57Z
Merge #49589 `merge svrcore into 389-ds-base`
- - - - -
5e3f428b by Mark Reynolds at 2018-03-07T23:50:02Z
Ticket 49596 - repl-monitor.pl fails to find db tombstone/RUV entry

Description:  The tool was doing a scope "one" search to get the tombstone/RUV
              entry.  This no longer works as it should be a "sub" search.

https://pagure.io/389-ds-base/issue/49596

Reviewed by: firstyear(Thanks!)

- - - - -
02a8defa by Ludwig Krispenz at 2018-03-08T12:19:59Z
missing clang formatting in urp_tombstone

- - - - -
b67f0f82 by Mark Reynolds at 2018-03-12T16:29:07Z
Ticket 49599 - Revise replication total init status messages

Description:  Previously the replication update status messages were
              updated to a more readable format.  This should also be
              done with the total update status messages.

https://pagure.io/389-ds-base/issue/49599

Reviewed by: lkrispen(Thanks!)

- - - - -
0f0725c9 by Simon Pichugin at 2018-03-13T07:43:59Z
Issue 49593 - NDN cache stats should be under the global stats

Bug description: The Normalized DN cache stats are listed under
backend stats, but the cache is global across all backends so
they should just be listed under the global ldbm monitor.

Fix description: Move NDN stats to global ldbm monitor.
Change lib389 MonitorLDBM and MonitorBackend objects accordingly.
Fix dbmon.sh tool so it shows the stats like this:
Wed Mar  7 12:50:14 EST 2018
dbcachefree 52150272 free% 99.812 roevicts 0 hit% 100 pagein 0 pageout 0
      dbname      count          free  free%    size  hit_ratio
  global:ndn         69      20960096 99.946 165.565         73
userroot:ent          2     201318969  100.0  3811.5       62.0
userroot:dn           2      67108738  100.0    63.0        0.0

https://pagure.io/389-ds-base/issue/49593

Reviewed by: mreynolds, wibrown (Thanks!)

- - - - -
b05fe2fe by Viktor Ashirov at 2018-03-16T14:49:22Z
Issue 49603 - 389-ds-base package rebuilt on EPEL can't be installed due to missing dependencies

Bug Description:
On EPEL python3 packages are not built, but still mentioned in Requires.
This prevents 389-ds-base from being installed.

Fix Description:
Require python3-lib389 only on Fedora and EPEL > 7. Additionally fix
conditionals according to packaging guidelines.

https://pagure.io/389-ds-base/issue/49603

Reviewed by: mreynolds (Thanks!)

- - - - -
2020610d by Mark Reynolds at 2018-03-19T15:13:39Z
Ticket 49552 - Fix build issues on F28

Description: Builds started breaking on F28.  Removed part of the last fix to
             address previous build issues on F28.  The fix was just to remove the
             libslapd libary from libldaputil.  It was never needed in the first
             place.

https://pagure.io/389-ds-base/issue/49552

Reviewd by: mreynolds (one line commit rule)

- - - - -
56e064ef by Simon Pichugin at 2018-03-20T07:22:00Z
Issue 49606 - Improve lib389 documentation

Description: Refactor existing chapters and add more docstrings
to existing modules. Also, make sure that the contribution guide is
up to date with the new pull-request initiative.

https://pagure.io/389-ds-base/issue/49606

Reviewed by: wibrown (Thanks!)

- - - - -
5ba01818 by Viktor Ashirov at 2018-03-20T23:51:55Z
Issue 49608 - Add support for gcc/clang sanitizers

GCC and Clang provide various sanitizers:
Address sanitizer: https://clang.llvm.org/docs/AddressSanitizer.html
Memory sanitizer (Clang only): https://clang.llvm.org/docs/MemorySanitizer.html
Thread sanitizer: https://clang.llvm.org/docs/ThreadSanitizer.html
Undefined behaviour: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html

See also: https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html

This commit adds new configure options:
--enable-msan - enable memory sanitizer
--enable-tsan - enable thread sanitizer
--enable-ubsan - enable undefined behaviour sanitizer

They can't be used together, only one at a time.

https://pagure.io/389-ds-base/issue/49608

Reviewed by: wibrown (Thanks!)

- - - - -
5bfe2a3e by Thierry Bordaz at 2018-03-23T14:58:14Z
Ticket 49619 - adjustment of csn_generator can fail so next generated csn can be equal to the most recent one received

Bug Description:
	On consumer side csn_generator ajustment occurs (let CSN = highest known csn)

	    when a replication session starts
	    when a csn is generated locally and than csn is <= CSN

	During adjustment, in the case

	    there is no remote/local offset (time change)
	    the current_time on the consumer is identical to CSN

	Then next locally generated csn will only differ with seqnum

	The seqnum of the csn_generator is increased only if CSN.seqnum is larger
	than the csn_generator one.
	In case of egality, it remains unchanged.

	The consequence is that the next locally generated csn will be identical to CSN (except for the RID).
	So even after csn_generator adjustment, csn_generator may create csn that are not larger than the CSN

Fix Description:
	compare the new generated timestamp (time+offsets) with adjustment one.
	If the new is greater or EQUAL, make sure the local seqnum is ahead the remote one

https://pagure.io/389-ds-base/issue/49619

Reviewed by: Mark Reynolds

Platforms tested: F27

Flag Day: no

Doc impact: no

- - - - -
a3d3e1a9 by Matúš Honěk at 2018-03-26T15:44:15Z
Ticket 49601 - Replace HAVE_SYSTEMD define with WITH_SYSTEMD in svrcore

Bug Description: As former configure of svrcore is not used after the
merge of svrcore, and the svrcore's --with-systemd configure flag
handling seems to not have been merged to the 389-ds-core configure,
and configuring svrcore --with-systemd defines HAVE_SYSTEMD which is
different from define WITH_SYSTEMD used in 389-ds-base, the systemd
support in svrcore is not effectively turned on when compiling
389-ds-base with --with-systemd.

Fix Description: Use the very same define as 389-ds-base
uses (WITH_SYSTEMD) instead of the former one of svrcore, which is
obviously inheritted into the svrcore code as well, thus effectively
selecting a code to be compiled based on the --with-systemd configure
flag.

https://pagure.io/389-ds-base/issue/49601

Author: mhonek

Review by: mreynolds, spichugi

- - - - -
5e383c81 by Mark Reynolds at 2018-03-28T18:28:42Z
Ticket 46918 - Fix compiler warnings on arm

Description:  On "arm" architecture there were many compiler warnings
              and after fixing some it also adressed a crash in replication
              when trying to update the agreement maxcsn.

              All updatream architectures build without compilar warnings,
              and coverity scan did not report any regressions.

https://pagure.io/389-ds-base/issue/49618

Reviewed by: simon(Thanks!)

- - - - -
35523123 by Akshay Adhikari at 2018-03-29T08:50:10Z
Issue 49585 - Add py3 support to password test suite

Description: Added py3 support by explicitly changing strings to bytes.

https://pagure.io/389-ds-base/issue/49585

Reviewed by: spichugi

- - - - -
fda63dc5 by Mark Reynolds at 2018-03-29T17:52:56Z
Ticket 48184 - revert previous patch around unuc-stans shutdown crash

https://pagure.io/389-ds-base/issue/48184

- - - - -
84c9e750 by Akshay Adhikari at 2018-04-02T08:03:07Z
Issue 49585 - Add py3 support to password test suite : part-2

Description: Added py3 support by explicitly changing strings to bytes.

https://pagure.io/389-ds-base/issue/49585

Reviewed by: spichugi

- - - - -
0a5e39fb by Akshay Adhikari at 2018-04-04T13:16:44Z
Issue 49585 - Add py3 support to password test suite : part-3

Description: Added code to handle the value 'DEFAULT' in passwordStorageScheme
             for older version of DS.

https://pagure.io/389-ds-base/issue/49585

Reviewed by: vashirov

- - - - -
dbc4a248 by Ludwig Krispenz at 2018-04-05T06:58:22Z
Ticket 49631 - same csn generated twice

Bug: if in the csn adjustment the local time was less or equal than the remote time
     the sequence number has always been adjusted to remote++
     but if the csn time was equal and the local seq number was larger the effect
     was a reset of the csn generato.

Fix: correctly handles seqnum in csn adjustment

Reviewed by: Mark, thanks

- - - - -
ab35fe80 by Mark Reynolds at 2018-04-05T17:15:05Z
Ticket 49522 - Fix build issues on F28

Bug Description:  We are randomly get build failures around libldaputil.la

    libtool:   error: cannot find the library 'libldaputil.la' or
                      unhandled argument 'libldaputil.la'

Fix Description:  A previous commit introduced one of the issues, this patch
                  is adding libslapd.la to the LIBADD of libldaputil

https://pagure.io/389-ds-base/issue/49552

Reviewed by: mreynolds(one line commit rule)

- - - - -
c1d38ca5 by Mark Reynolds at 2018-04-06T21:14:23Z
Ticket 49552 - Fix the last of the build issues on F28/29

Description: Add missing  NSPR/NSS libs for libsrvcore, and add libldaputil
             to ns-slapd dependancies.

             Thanks to Matus for figuring outthe ns-slapd dependacnies part!!

https://pagure.io/389-ds-base/issue/49552

Reviewed by: mreynolds & mhonek (one line commit rule)

- - - - -
77357a2c by William Brown at 2018-04-11T04:23:48Z
Ticket 49477 - Missing pbkdf python

Bug Description:  When I wrote the tests for the pbkdf2 regression
I missed adding a file.

Fix Description:  Recreate and add the file as required.

https://pagure.io/389-ds-base/issue/49447

Author: firstyear

Review by: mreynolds (Thank you!)

- - - - -
872e4087 by Mark Reynolds at 2018-04-13T14:00:01Z
Bump version to 1.4.0.7

- - - - -
71cbd1d6 by Simon Pichugin at 2018-04-17T06:34:26Z
Issue 49511 - memory leak in pwdhash

Bug description: When we have multiple inputs
we leak 1 to N strings allocate.

Fix description: We free only one string in the end of main()
but if we have a multiple string inputs we should free
all of them during the for loop.

https://pagure.io/389-ds-base/issue/49511

Reviewed by: mreynolds (Thanks!)

- - - - -
e306a2d4 by Akshay Adhikari at 2018-04-18T09:52:51Z
Issue 49586 - Add py3 support to plugins test suite

Description: Added py3 support by explicitly changing strings to bytes.
Added code for creating a new connection to make ldap communicate on localhost.

https://pagure.io/389-ds-base/issue/49586

Reviewed by: spichugi

- - - - -
ea033b6b by Simon Pichugin at 2018-04-18T14:42:38Z
Issue 49109 - nsDS5ReplicaTransportInfo should accept StartTLS as an option

Bug Description: nsDS5ReplicaTransportInfo SSL vs TLS is not really clear,
given that most libraries now support TLS as the default "SSL".
We should make this clear in nsDS5ReplicaTransportInfo by allowing:
ldaps -> SSL
StartTLS -> TLS
options. So that it's really clear what you are asking for when you configure it.

Fix Description: Add additional options for nsDS5ReplicaTransportInfo - LDAPS and StartTLS.
Legacy options will stay for some time and will deprecated in later version.
Also, change the DNA plugin values in the same way.
And rename replica flags through the code base as followed:
TRANSPORT_FLAG_SSL -> TRANSPORT_FLAG_LDAPS
TRANSPORT_FLAG_TLS -> TRANSPORT_FLAG_STARTTLS
Also, check the change in the existing TLS replication test suite.

https://pagure.io/389-ds-base/issue/49109

Reviewed by: wibrown, mreynolds (Thanks!)

- - - - -
28f69b39 by Mark Reynolds at 2018-04-19T20:00:35Z
Ticket 49639 - Crash when failing to read from SASL conn

Description:  This is a regression from ticket 49618, a return code
              integer was reset to a unsigned int, when it needed to
              remain signed.  This allowed an error condition to go
              unchecked, which leads to a crash caused by a large
              realloc attempt from the overflowed integer result code.

https://pagure.io/389-ds-base/issue/49639

Reviewed by: mreynolds(one line commit rule)

- - - - -
1e0b18a0 by Mark Reynolds at 2018-04-19T21:34:11Z
Bump version to 1.4.0.8

- - - - -
5bfc18a2 by Mark Reynolds at 2018-04-20T15:05:25Z
Ticket 49644 - crash in debug build

Description:  In a debug build of the server it crashes when searching
              the cn=config.  This is due to a pointer not being initialized
              before being dereferenced.

https://pagure.io/389-ds-base/issue/49644

Reviewed by: mreynolds(one line commit rule)

- - - - -
bc181f6c by Simon Pichugin at 2018-04-20T15:28:45Z
Issue 49612 - lib389 remove_ds_instance() does not remove systemd units

Bug description: When running remove_ds_instance(), it does not remove
the systemd files that identify this instance.
Also, in previous DS versions, the /etc/dirsrv/slapd-INSTANCE
would have been moved to /etc/dirsrv/slapd-INSTANCE.removed.
This does not happen now.

Fix description: Change remove_ds_instance function so it disables systemd unit
and, in the result, it will remove symlinks. Also make the function
move the config_dir to config_dir.removed as remove-ds.pl does.
Add a basic test to src/lib389/lib389/tests/instance/remove_test.py
Fix local_simple_allocate, so it sets serverid properly.

https://pagure.io/389-ds-base/issue/49612

Reviewed by: mreynolds (Thanks!)

- - - - -
d357888d by Simon Pichugin at 2018-04-20T15:31:51Z
Issue 49642 - lib389 should generate a more complex password

Bug description: New NSS versions require a password to be more complex.
The password should be at least 7 characters long,
and should consist of at least three character classes.
The available character classes are: digits (0-9), ASCII
lowercase letters, ASCII uppercase letters, ASCII
non-alphanumeric characters, and non-ASCII characters.

Fix description: Refactor password_generate function
so it fullfils the minimal requirements. Also, make
the fuction generate more secure password.
Add the docstrings to the passwd.py module.

https://pagure.io/389-ds-base/issue/49642

Reviewed by: mreynolds (Thanks!)

- - - - -
530a2db1 by Mark Reynolds at 2018-04-24T15:00:01Z
Ticket 49649 - Use reentrant crypt_r()

Bug Description:  We were previously using crypt() which is not
                  thread safe and reuired a lock.  Using pwdhash cli
                  tool caused a crash because the lock was not created
                  when invoked by the cli.

Fix Description:  Use crypt_r() instead which does not require any locking.

https://pagure.io/389-ds-base/issue/49649

Reviewed by: Simon(Thanks!)

- - - - -
c9632e86 by Akshay Shivekar at 2018-04-25T09:02:13Z
Issue 49406 - Port backend_test.py test to DSLdapObject implementation

Description: Refactor old ones and add fixture for them.
Add the test cases for lint, create_sample_entries, get_monitor,
and get_indexes methods.

The commit was fixed by Simon Pichugin according to William Brown comments.

https://pagure.io/389-ds-base/issue/49406

Reviewed by: spichugi, wibrown (Thanks!)

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
a86ff68e by Simon Pichugin at 2018-04-25T11:40:59Z
Issue 49538 - replace cacertdir_rehash with openssl rehash

Bug description: Enable TLS method fails on newer Fedora systems.
It complains that cacertdir_rehash tool is not found.

Fix description: The reason is that authconfig was replaced with
authselect package. Authconfig shipped a tool called cacertdir_rehash
which is no longer available on Fedora. We need to switch to openssl tool:
'c_rehash <directory>' that serves the same purpose and is present on both
RHEL7 and Fedora.

Remove authconfig from 389-ds-base.spec.in because 'cacertdir_rehash'
was the only reason why we had the dependency.
Add openssl-perl dependency to the SPEC file.

Eventially, c_rehash will go away but for now we keep it for RHEL7 compatibility.

Fix a small thing in remove_ds_instance function.
We should ignore FileNotFound errors while removing the instance.

https://pagure.io/389-ds-base/issue/49538

Reviewed by: mreynolds, mhonek (Thanks!)

- - - - -
d2147659 by Matúš Honěk at 2018-04-26T10:42:05Z
Ticket 49650 - lib389 enable_tls doesn't work on F28

Bug Description: In the lib389 we have the method inst.enable_tls().
It creates certificates and sets up the server for TLS communication.
It works on F27 built from master branch and doesn't work on F28.
It happens because on F28 openssl fails to verify the certificate.

Fix Description: We should create CA with an appropriate flag.
It can be done by setting an X.509 V3 Certificate Type Extension
in the certificate to 'certSigning' value.

https://pagure.io/389-ds-base/issue/49650

Author: mhonek

Review by: spichugi, wibrown (Thanks!)

- - - - -
d77c7f07 by Mark Reynolds at 2018-04-27T12:36:06Z
Ticket 49652 - DENY aci's are not handled properly

Bug Description:  There are really two issues here.  One, when a resource
                  is denied by a DENY aci the cached results for that resource
                  are not proprely set, and on the same connection if the same
                  operation repeated it will be allowed instead of denied because
                  the cache result was not proprely updated.

                  Two, if there are no ALLOW aci's on a resource, then we don't
                  check the deny rules, and resources that are restricted are
                  returned to the client.

Fix Description:  For issue one, when an entry is denied access reset all the
                  attributes' cache results to DENIED as it's possible previously
                  evaluated aci's granted access to some of these attributes which
                  are still present in the acl result cache.

                  For issue two, if there are no ALLOW aci's on a resource but
                  there are DENY aci's, then set the aclpb state flags to
                  process DENY aci's

https://pagure.io/389-ds-base/issue/49652

Reviewed by: tbordaz & lkrispenz(Thanks!!)

- - - - -
9d8d096b by Mark Reynolds at 2018-05-08T14:48:02Z
CVE-2018-1089 - Crash from long search filter

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
5bf4fd58 by Mark Reynolds at 2018-05-08T15:08:59Z
Bump version to 1.4.0.9

- - - - -
dc690dd2 by Mark Reynolds at 2018-05-09T20:36:48Z
Ticket 49665 - Upgrade script doesn't enable PBKDF2 password storage plug-in

Description:  There is no upgrade script to add the PBKDF2 plugin, this
              fix adds the script.

https://pagure.io/389-ds-base/issue/49665

Reviewed by: ?

- - - - -
91dc8324 by Mark Reynolds at 2018-05-10T12:00:17Z
Ticket 49665 - Upgrade script doesn't enable CRYPT password storage plug-in

Description:  There is no upgrade script to add the new CRYPT plugins, this
              fix adds the script.

https://pagure.io/389-ds-base/issue/49665

Reviewed by: vashirov(Thanks!)

- - - - -
3a70d8a4 by Mark Reynolds at 2018-05-10T13:56:29Z
Ticket 49519 - Convert Cockpit UI to use strictly patternfly stylesheets

Description:  I had previously overwritten most of PF stylesheets.  This
              patch contains that conversion.

https://pagure.io/389-ds-base/issue/49519

- - - - -
5d700cc4 by Mark Reynolds at 2018-05-11T19:01:18Z
Ticket 49673 -  nsslapd-cachememsize can't be set to a value bigger than MAX_INT

Bug Description:  Regression from ticket https://pagure.io/389-ds-base/issue/49618
                  This ticket accidentally set a MAX_INT cap on the entry cachesize.

Fix Description:  Set proper max value allowed for entry and dn caches (uint64)

https://pagure.io/389-ds-base/issue/49673

Reviewed by: mreynolds(one line commit rule)

- - - - -
afb755bd by Mark Reynolds at 2018-05-13T19:01:16Z
Ticket 49671 - Readonly replicas should not write internal ops to changelog

Bug Description:  When a hub receives an update that triggers the memberOf
                  plugin, but that interal operation has no csn and that
                  causes the update to the changelog to fail and break
                  replication.

Fix Description:  Do not write internal updates with no csns to the changelog
                  on read-only replicas.

https://pagure.io/389-ds-base/issue/49671

Reviewed by: simon, tbordaz, and lkrispen (Thanks!!!)

- - - - -
50732c9e by Simon Pichugin at 2018-05-14T07:33:23Z
Issue 49657 - Fix cascading replication scenario in lib389 API

Bug description: We should be able to create cascading replication topology
with existing lib389 API. Fix existing topology in topologies.py
and add the functionality to replica.py API accordingly.

Fix description: Add the code to join_hub function.
Get agreement name from the last three port numbers.
When we call _create_service_group on read-only instance
do not try to create it.
Generate the replica credentials when adding the service account
and store them in ReplicationManager object.
Fix a few small issues in replica.py.
In topologies.py, divide the instance and replica creation.
Refactor topology_m1h1c1 according to the changes.
During the finalizer, check if instance exists before the removal.

https://pagure.io/389-ds-base/issue/49657

Reviewed by: mreynolds (Thanks!)

- - - - -
6e1ee76e by Viktor Ashirov at 2018-05-14T11:20:45Z
Issue 49106 - Move ds_* scripts to libexec

Bug Description:
Binaries like
/usr/sbin/ds_selinux_enabled
/usr/sbin/ds_selinux_port_query
/usr/sbin/ds_systemd_ask_password_acl

are not user-runnable, they are executed by other programs (setup-ds.pl
for example). They should not reside in /usr/sbin, since it's used for
storing binaries for system administration. Instead they should be placed
in /usr/libexec/dirsrv/ which is designed to store binaries that are
executed by other programs.

Fix Description:
Change install path to libexec.

https://pagure.io/389-ds-base/issue/49106

Reviewed by: mhonek, wibrown (Thanks!)

- - - - -
3037674a by Viktor Ashirov at 2018-05-14T14:00:35Z
Issue 49685 - make clean fails if cargo is not installed

Bug Description:
`make clean` fails if cargo is not installed.

Fix Description:
Call `cargo clean` only when we enable Rust.

https://pagure.io/389-ds-base/issue/49685

Reviewed by: mreynolds (Thanks!)

- - - - -
68ecbff0 by Viktor Ashirov at 2018-05-14T14:07:44Z
Issue 49684 - AC_PROG_CC clobbers CFLAGS set by --enable-debug

Bug Description:
AC_PROG_CC macro sets CFLAGS/CXXFLAGS environment variable to ` -g -O2`
for GCC if CFLAGS/CXXFLAGS are not defined.

Fix Description:
Use an empty default for CFLAGS and CXXFLAGS instead.

https://pagure.io/389-ds-base/issue/49684

Reviewed by: mreynolds (Thanks!)

- - - - -
15ff2e3d by Mark Reynolds at 2018-05-15T13:21:34Z
Ticket 49669 - Invalid cachemem size can crash the server during a restore

Bug Description:  If you manually set the dbcachememsize to something larger than
                  a uint64_t the server can crash from a NULL pointer being
                  dereferenced.

Fix Description:  Catch the NULL pointer before it is dereferenced, and abort the
                  restore.

https://pagure.io/389-ds-base/issue/49669

Reviewed by: firstyear & tbordaz (Thanks!!)

- - - - -
0282ef26 by Ludwig Krispenz at 2018-05-17T09:38:01Z
    Ticket 49696: replicated operations should be serialized

    Bug: there was a scenario where two threads could process replication operations in parallel.
         The reason was that for a new repl start request the repl conn flag is not set and the
         connection is made readable.
         When the start repl op is finished, the flagi set, but in a small window the supplier could
         already have sent updates and more_data would trigger this thread also to continue to process
         repl operations.

    Fix: In the situation where a thread successfully processed a start repl request and just set the repl_conn
         flag  do not use more_data.

    Reviewed by: Thierry, thanks

- - - - -
6157c6a8 by Thierry Bordaz at 2018-05-17T10:47:20Z
Ticket 49693 - A DB_DEADLOCK while adding a tombstone (RUV) leads to access of an already freed entry

Bug Description:
	During a ADD, in order to manage DB_DEADLOCK, instead of using the entry provided in the pblock
	(i.e. 'e') the code uses a couple addingentry/originalentry.
	Only in the initial attempt addingentry refers to 'e', in the others it refers to a duplicate one.
	On DB_DEADLOCK, the entry is freed immediately (as it was not in the cache)

	if we hit a DB_DEADLOCK then 'e' is freed and the next attempt is with a duplicate of 'e'.
	But if the added entry is a tombstone we log a message dumping 'e', unfortunately 'e' was already freed.

Fix Description:
	Use addingentry->ep_entry instead of 'e'. Also as it is for logging, test if the logging
	level is set before dumping the entry.

https://pagure.io/389-ds-base/issue/49693

Reviewed by: Ludwig Krispenz, Mark Reynolds (thanks !!)

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
70314dc2 by Mark Reynolds at 2018-05-17T10:58:28Z
Ticket 49665 - remove obsoleted upgrade scripts

Description:  Remove the recently added upgrade scripts as they are
              obsolete in 1.4.0.

https://pagure.io/389-ds-base/issue/49665

Reviewed by: ?

- - - - -
dc7f1083 by Simon Pichugin at 2018-05-17T12:55:35Z
Issue 49581 - Fix dynamic plugins test suite

Description: Refactor plugins module and fix tests accordingly.
Divide the tests into three functions: accaptance, memory corrruption
and stress.  Add absent Task and Plugin objects.
Add docstrings for every fixed object.
Move plugin acceptance tests to a separate module in plugins suite

https://pagure.io/389-ds-base/issue/49581

Reviewed by: vashirov, mreynolds, wibrown (Thanks!)

- - - - -
4521fe33 by Mark Reynolds at 2018-05-18T13:58:29Z
Ticket 49698 - Remove unneeded patternfly files from Cockpit package

Description: Remove files from Pattern we do not use in order to help reduce its footprint
             on the package

https://pagure.io/389-ds-base/issue/49698

Reviewed by: mreynolds(one line commit rule)

- - - - -
e562157c by Thierry Bordaz at 2018-05-18T16:16:56Z
Ticket 48184 - clean up and delete connections at shutdown (2nd try)

Bug description:
    During shutdown we would not close connections.
    In the past this may have just been an annoyance, but now with the way
    nunc-stans works, io events can still trigger on open xeisting connectinos
    during shutdown.

    Because of NS dynamic it can happen that several jobs wants to work on the
    same connection. In such case (a job is already set in c_job) we delay the
    new job that will retry.
    In addition:
	- some call needed c_mutex
	- test uninitialized nunc-stans in case of shutdown while startup is not completed

Fix Description:  Close connections during shutdown rather than
    leaving them alive.

https://pagure.io/389-ds-base/issue/48184

Reviewed by:
	Original was Ludwig and Viktor
	Second fix reviewed by Mark

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
53e58cdb by Mark Reynolds at 2018-05-18T16:31:04Z
Ticket 49576 - Update ds-replcheck for new conflict entries

Description:  This patch addresses the recvent changes to conflict
              entries and tombstones.

https://pagure.io/389-ds-base/issue/49576

Reviewed by: tbordaz(Thanks!)

- - - - -
262e6aaf by Akshay Adhikari at 2018-05-21T09:22:34Z
Issue 49588 - Add py3 support for tickets : part-1

Description: Added py3 support by explicitly changing strings to bytes.
Ported tests from ticket to test suites, also added docstrings.

https://pagure.io/389-ds-base/issue/49588

Reviewed by: spichugi,vashirov (Thanks!)

- - - - -
9ff284fc by Viktor Ashirov at 2018-05-21T11:05:21Z
Issue 49679 - Missing nunc-stans documentation and doxygen warnings

Add missing nunc-stans documentation from the old repo. This also should
fix doxygen warnings about missing files

https://pagure.io/389-ds-base/issue/49679

Reviewed by: mreynolds (Thanks!)

- - - - -
50b54494 by Mark Reynolds at 2018-05-21T17:47:09Z
Ticket 49689 - Move Cockpit UI plugin to a subpackage

Description:  Move the new UI plugin to a subpackage of 389-ds-base.
              Special thanks to Viktor Ashirov for helping on this one!

https://pagure.io/389-ds-base/issue/49689

Reviewed by: vashirov(Thanks!)

- - - - -
efc96f10 by Mark Reynolds at 2018-05-22T16:27:50Z
Ticket 49689 - Fix local "make install" after adding cockpit subpackage

Bug Description:  When doing a local "make install" the cockpit UI files
                  are not copied to the "buildroot", which then leads to
                  rsync failing.

Fix Description:  If the "source directory" is not the same as the current
                  directory, then its a local "make install" and not a
                  "make rpms".  In that case just copy over the cockpit ui
                  directory to the local buildroot.  This makes it easy to
                  test Cockpit UI changes using "make install".

https://pagure.io/389-ds-base/issue/49689

Reviewed by: mreynolds(one line commit rule)

- - - - -
c1e022d4 by Viktor Ashirov at 2018-05-23T14:01:15Z
Issue 49678 - organiSational vs organiZational spelling in lib389

Bug description:
RFC4519 defines 'organization' object class using Oxford spelling.
It is common for academic, formal, and technical writing for an
international readership. We should be following common spelling
in function names and methods, otherwise it is confusing
and annoying to remember which one is used where.

Fix description:
Use Oxford spelling for 'organization' and 'organizationalUnit'.

https://pagure.io/389-ds-base/issue/49678

Reviewed by: spichugi, mhonek (Thanks!)

- - - - -
8330887e by Viktor Ashirov at 2018-05-23T14:11:29Z
Issue 49684 - AC_PROG_CC clobbers CFLAGS set by --enable-debug

Bug description:
Default CFLAGS and CXXFLAGS might be unset without --enable-debug.

Fix description:
* Provide default CFLAGS and CXXFLAGS that would be set by AC_PROG_CC
otherwise.
* Split compiler flags and preprocessor flags into separate variables so
they are applied in a correct order.

https://pagure.io/389-ds-base/issue/49684

Reviewed by: mhonek (Thanks!)

- - - - -
164a9591 by Mark Reynolds at 2018-05-23T18:46:14Z
Ticket 49706 - Finish UI patternfly convertions

Description:  A few modals were not converted to PF, and other minor issues
              with JS and updating tables

https://pagure.io/389-ds-base/issue/49706

Reviewed by: ?

- - - - -
9e046a35 by Mark Reynolds at 2018-05-25T13:47:31Z
Ticket 49576 - Add support of ";deletedattribute" in ds-replcheck

Description: Also need to check for ";deletedattribute" when processing LDIF file

https://pagure.io/389-ds-base/issue/49576

Reviewed by: tbordaz(Thanks!)

- - - - -
7a8b5ace by Mark Reynolds at 2018-05-25T16:26:55Z
Ticket 49675 - Fix coverity issues

Description:  Fixed these coverity issues.  Some of these fixes are
              just to quiet convscan:

16852   Unsigned compared - entrycache_add_int
16848   Unsigned compared - dncache_add_int
16704   Explicit null dereferenced s- lapd_SSL_client_auth
15953   Resource leak - new_task
15583   Out-of-bounds read - create_filter
15445   Unused value - ruv_update_ruv
15442   Argument cannot be negative - dse_write_file_nolock
15223   Double unlock - ruv_get_referrals
15170   Explicit null dereferenced - passwd_apply_mods
15581   Wrong sizeof argument - slapi_be_new
15144   Constant expression result - upgradedn_producer

               Also fixed a few compiler warnings

https://pagure.io/389-ds-base/issue/49675

Reviewed by: spichugi & lkrispenz(Thanks!!)

- - - - -
1d9a80fb by Anuj Borah at 2018-05-28T06:48:31Z
Issue 49582 - Add py3 support to memberof_plugin test suite

Description: Fix memberof_plugin/regression_test.py by switching to ReplicationManager.

https://pagure.io/389-ds-base/issue/49582

Reviewed by: vashirov

- - - - -
e350a268 by Mark Reynolds at 2018-05-28T14:09:58Z
Ticket 49722 - Errors log full of " WARN - keys2idl - recieved NULL idl from index_read_ext_allids, treating as empty set" messages

Description:  If searching on entrydn, and the value is not found return an empty
              list instead of NULL.  This prevent these harmless error messages in
              log

https://pagure.io/389-ds-base/issue/49722

Reviewed by: ?

- - - - -
27a16a06 by Mark Reynolds at 2018-05-28T15:42:39Z
Ticket 49726 - DS only accepts RSA and Fortezza cipher families

Bug Description:  Currently DS only accepts fortezza and RSA cipher families.
                  This prevents things like ECC certificates from being used.

Fix Description:  Instead of hardcoding the cipher families, just grab the
                  current type and use it.

                  Also cleaned up code: removed unncessary "ifdefs", and switched
                  for loops to use size_t.

https://pagure.io/389-ds-base/issue/49726

Reviewed by: ?

- - - - -
6bfe74fc by Mark Reynolds at 2018-05-28T16:10:23Z
Ticket 49704 - Error log from the installer is concatenating all lines into one

Description:  Created new bootpopup wrapper functions that use the "content" option
              to add html messages.  Then we use "pre" html tags for messages that
              come from the CLI, and regular "p" tags for generic messages.

              Also durign instance creation we now strip "slapd-" from instance
              name because the installer automatically adds "slapd-" to the name
              anyway.

https://pagure.io/389-ds-base/issue/49704

Reviewed by: firstyear(Thanks!)

- - - - -
593a73e6 by Mark Reynolds at 2018-05-29T20:08:22Z
Ticket 49683 - Add support for JSON option in lib389 CLI tools

Description:  This is the initial patch for adding a json option to
              the CLI tools. This patch also addresses issues with
              using LDAPI with the CLI.

https://pagure.io/389-ds-base/issue/49683

Reviewed by: firstyear & spichugi (Thanks!!)

- - - - -
2e467029 by Matúš Honěk at 2018-05-30T11:52:16Z
Ticket 49333 - Do not remove versioned man pages

Bug Description: Until resolving ticket 49689 in 50b5449, running
`make clean' removed also man pages which were tracked in the
versioning system (notably man1 and man8).

Fix Description: Bring back only removal of man3 man pages as they are
autogenerated and thus should not be tracked.

https://pagure.io/389-ds-base/issue/49333

Author: mhonek

Review by: mreynolds (thanks!)

- - - - -
1b7198a0 by Mark Reynolds at 2018-05-30T17:13:59Z
Ticket 49675 - Revise coverity fix

Description:  Fix issues with last coverity patch:  missing unlock, and a
              return code was needed.

              Also fixed issue 17472 (memory leak in uid.c)

https://pagure.io/389-ds-base/issue/49675

Reviewed by: tbordaz & lkrispenz(Thanks!!)

- - - - -
085e99f7 by Thierry Bordaz at 2018-05-31T16:34:06Z
Ticket 48184 - clean up and delete connections at shutdown (3rd)

Bug description:
        During shutdown we would not close connections.
        In the past this may have just been an annoyance, but now with the way
        nunc-stans works, io events can still trigger on open xeisting connectinos
        during shutdown.

Fix Description:
        Because of NS dynamic it can happen that several jobs wants to work on the
        same connection. In such case (a job is already set in c_job) we delay the
        new job that will retry.
        In addition:
            - some call needed c_mutex
            - test uninitialized nunc-stans in case of shutdown while startup is not completed

	If it is not possible to schedule immediately a job it is sometime useless to wait:
		- if the connection is already freed, just cancel the scheduled job
		  and do not register a new one
		- If we are in middle of a shutdown we do not know if the
		  scheduled job is ns_handle_closure, so cancel the scheduled
		  job and schedule ns_handle_closure.

https://pagure.io/389-ds-base/issue/48184

Reviewed by:
            Original fix reviewed by Ludwig and Viktor
            Second   fix reviewed by Mark
	    Third    fix reviewed by Mark

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
d09a57d4 by Mark Reynolds at 2018-05-31T19:28:25Z
Add Cockpit UI fonts

On F28 these fonts are requested by both Firefox and Chrome so I'm adding them back.

Reviewed by: mreynolds

- - - - -
b0e05806 by Thierry Bordaz at 2018-06-01T15:27:19Z
Ticket 49736 - Hardening of active connection list

Bug Description:
	In case of a bug in the management of the connection refcnt
	it can happen that there are several attempts to move a connection
	out of the active list.

	It triggers a crash because when derefencing c->c_prev.
	c_prev is never NULL on the active list

Fix Description:
	The fix tests if the connection is already out of the active list.
	If such case, it just returns.

	A potential issue that is not addressed by this fix is:
	Thread A and Thread B are using 'c' but c->refcnt=1 (it should be 2)
	Thread A "closes" 'c', 'c' is move out of active list (free) because of refcnt=0
	A new connection happens selecting the free connection 'c', moving it to the active list.
	Thread C is using 'c' from the new connection c->refcnt=1
	Thread B "closes" 'c', 'c' is moved out of the active list.
	-> new operation coming on 'c' will not be detected
	-> Thread C will likely crash when sending result

https://pagure.io/389-ds-base/issue/49736

Reviewed by: Mark Reynolds (thanks!)

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
f6d94228 by Timo Aaltonen at 2018-06-02T06:32:46Z
Merge tag '389-ds-base-1.3.8.2' into master-next

- - - - -
397bd7f8 by Timo Aaltonen at 2018-06-02T06:32:53Z
Merge branch 'master' into master-next

- - - - -
b4767203 by Timo Aaltonen at 2018-06-02T06:34:57Z
update the changelog

- - - - -
28dac5e0 by Timo Aaltonen at 2018-06-02T06:39:12Z
control: Update maintainer address.

- - - - -
4a7c3632 by Timo Aaltonen at 2018-06-02T06:43:38Z
control: Upstream dropped support for non-64bit architectures, so build only on supported 64bit archs (amd64, arm64, mips64el, ppc64el, s390x).

- - - - -
bffa315b by Timo Aaltonen at 2018-06-02T06:50:00Z
control: Drop 389-ds metapackage, the other 389-ds-* packages are EOL'd upsream and will be removed soon.

- - - - -
39415596 by Timo Aaltonen at 2018-06-02T06:51:17Z
control: svrcore got merged here, drop it from build-depends.

- - - - -
d5e1164a by Mark Reynolds at 2018-06-04T13:44:30Z
Ticket 48377 - Bundle jemalloc

Descriptrion:  gperftools is going away in RHEL, that includes tcmalloc, so
               we now need to bundle jemalloc again.

https://pagure.io/389-ds-base/issue/48377

Reviewed by: vashirov(Thanks!)

- - - - -
67efef07 by Mark Reynolds at 2018-06-05T12:13:48Z
Ticket 49732 - Optimize resource limit checking for rootdn issued searches

Description:  When performing a search as the Directory Manager resource limits
              should not apply. So there is no need to "get" the limits if its
              a Directory Manager initiated search. I'm seeing around 2% performance
              increase when we skip getting the resource limits.

https://pagure.io/389-ds-base/issue/49732

Reviewed by: tbordaz (Thanks!)

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
d870eb0d by Mark Reynolds at 2018-06-05T15:44:35Z
Ticket 49748 - Passthru plugin startTLS option not working

Description:  While you can configure a connection to use StartTLS the
              plugin code did not attempt to use StartTLS.

https://pagure.io/389-ds-base/issue/49748

Reviewed by: tbordaz & lkrispenz(Thanks!!)

- - - - -
809be53d by Simon Pichugin at 2018-06-06T13:07:05Z
Issue 49646 - Improve TLS cert processing in lib389 CLI

Description: Add a new option to setup.inf file - self_sign_cert_valid_months
which accepts int value. dscreate should renew Self-signed CA if
it is expired or it will expire less than in 2 months.
Also, we need to import it to other existing instances.
Remove the NSS DB in the test topology during a teardown.
Fix small issues in nss_tls_test.
Add format_cmd_list function to utils.py. It returns a nicely formatted
quoted representation of the command list we put into subprocess call.
Add more logging to nss_tls.py. Change the default validity period to 24 months.

https://pagure.io/389-ds-base/issue/49646

Reviewed by: mreynolds, tbordaz, mhonek (Thanks!)

- - - - -
791e5aab by Simon Pichugin at 2018-06-06T20:50:04Z
Issue 49381 - Add docstrings to plugin test suites - Part 1

Description: Add and refactor the test case docstrings.
They should follow one exact format for communicating
with internal tool. Also it should be readable enough
to provide the main idea about the test case.

https://pagure.io/389-ds-base/issue/49381

Reviewed by: vashirov (Thanks!)

- - - - -
7c9ccb80 by Mark Reynolds at 2018-06-07T12:28:30Z
Ticket 47902 - UI - add continuous refresh log feature

Description:  Added checkbox to turn on server log continuous refresh

https://pagure.io/389-ds-base/issue/49702

Reviewed by: spichugi(Thanks!)

- - - - -
9d8887ad by Mark Reynolds at 2018-06-07T13:34:41Z
Ticket 49754 - instances created with dscreate can not be upgraded with setup-ds.pl

Bug Description:  If you create an instance with dscreate and try to
                  upgrade that instance using setup-ds.pl it will fail.
                  While you should not mix and match the python and perl
                  tools, it is still possible and it should work.

Fix Description:  The first problem was that the instance dir (inst_dir)
                  was not set during dscreate, and second we were also not
                  setting the correct inst_dir in defaults.inf.

                  Also dscreate does not create a backend by default,
                  which caused 80upgradednformat.pl to fail on an upgrade.

                  Finally updated UI's instance creation template to use
                  the correct inst_dir.

https://pagure.io/389-ds-base/issue/49754

Reviewed by: vashirov(Thanks!)

- - - - -
88950b0d by Simon Pichugin at 2018-06-07T13:37:35Z
Issue 49761 - Fix test suite issues

Description: A lot of tests are failing because of the API changes.
We need to fix them ASAP.

Fix description: Add __init__.py to import and mapping_tree test suite
so it will create its own __pycache__.
Use the new Agriments object for reinit.
Run range_search memory leak test only when ASAN is enabled and
remove valgrind support because ASAN is enough.
Generate ou=People and ou=Groups in the dbgen.py module, it is
required by some tests.
In the replica.py module, use existing credentials when we join master to
the existing first_master and  when we failed to get it from the ReplicationManager.

https://pagure.io/389-ds-base/issue/49761

Reviewed by: mreynolds (Thanks!)

- - - - -
dec51953 by Mark Reynolds at 2018-06-07T14:24:43Z
Ticket 49745 - UI add filter options for error log severity levels

Description:  Add a filtering option for error logging that allows
              for filtering log content by severity levels.  This includes
              a generic option for all "errors" and all "info messages"

https://pagure.io/389-ds-base/issue/49745

Reviewed by: spichugi(Thanks!)

- - - - -
cfb7dc2b by Thierry Bordaz at 2018-06-07T16:21:02Z
Ticket 49765 - Async operations can hang when the server is running nunc-stans

Bug Description:
	The fix https://pagure.io/389-ds-base/issue/48184 allowed to schedule
	several NS handlers where each handler waits for the dispatch of the
	previous handler before being schedule.

	In case the current handler is never called (connection inactivity)
	those that are waiting can wait indefinitely (until timeout or connection
	closure). But those that are waiting delay the processing of the operation
	when the scheduling is called by connection_threadmain.

	The some operations can appear hanging.
	This scenario happens with async operations

Fix Description:
	Instead of waiting for the completion of the scheduled handler,
	evaluates if the scheduled handler needs to be cleared (ns_job_done)
	or the waiting  handler to be canceled.

https://pagure.io/389-ds-base/issue/49765

Reviewed by: Mark Reynolds (thanks Mark !)

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
68cf69c8 by Mark Reynolds at 2018-06-08T12:10:53Z
Ticket 49689 - Cockpit subpackage does not build in PREFIX installations

Description:  For prefix isntallations we need to make sure the destination
              location directory tree is created:  mkdir -p

https://pagure.io/389-ds-base/issue/49689

Reviewed by: mreynolds(one line commit rule)

- - - - -
2a82ad2d by Thierry Bordaz at 2018-06-08T12:35:32Z
Ticket 49765 - compiler warning

- - - - -
8f04487f by Thierry Bordaz at 2018-06-08T13:13:51Z
Ticket 49768 - Under network intensive load persistent search can erronously decrease connection refcnt

Bug Description:
	If a connection enters in turbo mode (because of high traffic) or
	a worker reads several requests in the read buffer (more_data), the thread
	keeps processing connection.
	In that condition it should not decrease the refcnt.
	In case the operation is a persistent search, it decreases systematically
	the refcnt.
	So refcnt can become lower than the actual number of threads active on the connection.

	Most of the time it can create messages like
		Attempt to release connection that is not acquired
	In some rare case, if the a connection is out of the active list but a remaining thread
	tries to remove it again it can lead to a crash

Fix Description:
	The fix consist, when processing a PS, to decrease the refcnt at the condition
	the connection is not in turbo mode or in more_data.

https://pagure.io/389-ds-base/issue/49768

Reviewed by: Mark Reynolds

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
05587eda by Thierry Bordaz at 2018-06-08T15:56:03Z
Ticket 49742 - Fine grained password policy can impact search performance

Bug Description:
	new_passwdPolicy is called with an entry DN.
	In case of fine grain password policy we need to retrieve
	the possible password policy (pwdpolicysubentry) that applies to
	that entry.
	It triggers an internal search to retrieve the entry.

	In case of a search operation (add_shadow_ext_password_attrs), the
	entry is already in the pblock. So it is useless to do an additional
	internal search for it.

Fix Description:
	in case of fine grain password policy and a SRCH operation,
	if the entry DN matches the entry stored in the pblock (SLAPI_SEARCH_RESULT_ENTRY)
	then use that entry instead of doing an internal search

https://pagure.io/389-ds-base/issue/49742

Reviewed by: Mark Reynolds

Platforms tested: F26

Flag Day: no

Doc impact: no

- - - - -
e33c5013 by Mark Reynolds at 2018-06-08T16:24:14Z
Cockpit UI Fixes

This patch addresses these tickets:

    https://pagure.io/389-ds-base/issue/49741
      - Fixed repl agreement edit form

    https://pagure.io/389-ds-base/issue/49740
      - Set color-blind friendly colors.  Also used Thumbs up/down
        icons for repl state

    https://pagure.io/389-ds-base/issue/49703
      - Set default values in create instance form (also checks if
        default ports are available

Reviewed by: vashirov(Thanks!)

- - - - -
9186bcec by Mark Reynolds at 2018-06-08T16:29:42Z
Ticket 49571 - perl subpackage and python installer by default

Bug Description:  With 1.4.0 fast coming, we need to split
the legacy perl out to a subpackage, and coerce our tests to
python installer by default.

Fix Description:  Add a legacy tools subpackage

https://pagure.io/389-ds-base/issue/49571

Author: wibrown & vashirov(Thanks!!)

- - - - -
2dad1cd8 by Simon Pichugin at 2018-06-08T19:04:28Z
Issue 49640 - Errors about PBKDF2 password storage plugin at server startup

Bug description: Schema check errors are logged in the error log at start up.
It happens because of the missing attributes that are required by
nsslapdPlugin objectclass. Also we add the Password Storage Schemes
two times: in config.c and in fedse.c.

Fix description: Remove PBKDF2_SHA256 from slapd_bootstrap_config
because setup_internal_backends will happen shortly after (during main.c).

Another part of the fix - add nsslapd-pluginId, nsslapd-pluginVersion,
nsslapd-pluginVendor, and nsslapd-pluginDescription attributes to
fedse.c. The values don't matter because it will be rewritten shortly
after from pwd_storage_scheme_init functions (still during the startup).

https://pagure.io/389-ds-base/issue/49640

Reviewed by: mreynolds (Thanks!)

- - - - -
6175d376 by Mark Reynolds at 2018-06-08T19:21:21Z
Bump version to 1.4.0.10

- - - - -
db80854c by Mark Reynolds at 2018-06-08T23:05:45Z
Revert "Bump version to 1.4.0.10"

This reverts commit 6175d376258d2e5e819b82e88464127ccd9bcdfd.

- - - - -
afd83086 by Mark Reynolds at 2018-06-08T23:16:29Z
Ticket 46918 - Fix compiler warnings on arm

Description:  There were still a few lingering compiler errors

https://pagure.io/389-ds-base/issue/46918

Reviewed by: mreynolds(one line commit rule)

- - - - -
24c5c00f by Mark Reynolds at 2018-06-08T23:18:43Z
Bump version to 1.4.0.10

- - - - -
8807440b by Mark Reynolds at 2018-06-11T15:52:57Z
Ticket 49576 - ds-replcheck: fix certificate directory verification

Description:  The tool would crash if you attempted to use a certificate
              directory for conntacting replicas.

https://pagure.io/389-ds-base/issue/49576

Reviewed by: ?

- - - - -
0d7259fa by Mark Reynolds at 2018-06-11T17:01:01Z
Ticket 48204 - remove python2 from scripts

Description:  Change the @pythonexec@ macro to use python3

https://pagure.io/389-ds-base/issue/48204

Reviewed by: ?

- - - - -
f9d19b0b by Mark Reynolds at 2018-06-11T20:05:31Z
Remove old RHEL/fedora version checking from upstream specfile

- - - - -
60339ce8 by Akshay Adhikari at 2018-06-13T10:55:35Z
Issue 49588 - Add py3 support for tickets : part-2

Description: Added py3 support by explicitly changing strings to bytes.
Ported tests from ticket to test suites, also added docstrings.

https://pagure.io/389-ds-base/issue/49588

Reviewed by: spichugi (Thanks!)

- - - - -
5226bf99 by Mark Reynolds at 2018-06-14T00:08:17Z
Ticket 49712 - lib389 CLI tools should return a result code on failures

Description:  I've also included the work for 49775 in this patch since
              there was a lot of overldap.

              For dsctl functions we need to check for True and False in
              order to detect an error.  For dsconf & dsidm we need to
              catch exceptions.  Once an error is detected we return error
              code (1).

              The changes for 49775 was to use the default archive directory
              if one was not specified to db2bak, and use the default ldif
              location for db2ldif.  This how the old tools worked, no
              reason not to carry over this convenience.  Also the format
              used for the file name (Instance name + Date/Time) is the same
              as the old cli tools.

              Also did some pep8 cleanup.

https://pagure.io/389-ds-base/issue/49712

Reviewed by: spichugi(Thanks!)

- - - - -
5fe4e21a by Mark Reynolds at 2018-06-14T16:06:38Z
Ticket 49777 - add config subcommand to dsconf

Description:  Added a "config" subcomand to dsconf for getting and
              setting cn=config attributes.  Also fixed JSON issues.

https://pagure.io/389-ds-base/issue/49777

Reviewed by: spichugi(Thanks!)

- - - - -
2707e398 by Dj Padzensky at 2018-06-14T19:25:55Z
Fixing 4-byte UTF-8 character validation

- - - - -
a518313e by Dj Padzensky at 2018-06-17T17:53:42Z
Test for issue #49788

- - - - -
d590a1cd by Mark Reynolds at 2018-06-19T16:39:27Z
Bump version to 1.4.0.11

- - - - -
1c5831c0 by Mark Reynolds at 2018-06-21T12:49:12Z
Ticket 49734 - Fix various issues with Disk Monitoring

Bug Description:  The first issue what the internal represenation of
                  the default error code changed since the feature was
                  added.  This caused the disk monitoring thread to loop
                  and never actually attempt to stop the server.  The
                  other issue is that with nunc-stans g_set_shutdown()
                  is no longer stops the server.

Fix Description:  Change the defaulterror logging level to be more robust
                  and accept all the default level values.  Also, we needed
                  to free the rotated logs when we delete them.  Finally
                  add raise() to g_set_downdow(), and make sure set_shutdown()
                  does not overwrite the slapd_shutdown value.

https://pagure.io/389-ds-base/issue/49734

Reviewed by: tbordaz & spichugi(Thanks!)

- - - - -
d2c26d89 by Mark Reynolds at 2018-06-21T16:38:32Z
Ticket 49751 - passwordMustChange attribute is not honored by a RO consumer if using "Chain on Update"

Bug Description:  For a chained bind request we do not check password
                  policy, in particular if the password must be reset.

Fix Description:  For a chained bind request check if the password must
                  be reset.  This is the only password policy setting that
                  must be applied on the connection itself.  All other
                  password policy is still enforced correctly.

https://pagure.io/389-ds-base/issue/49751

Reviewed by: spichugi, tbordaz, and lkrispen(Thanks!!!)

- - - - -
87876bca by Samuel Rakitničan at 2018-06-23T07:17:28Z
Update Source0 URL in rpm/389-ds-base.spec.in
- - - - -
749b9f33 by Mark Reynolds at 2018-06-25T21:39:54Z
Ticket 49471 - Rename dscreate options

Description:  Changed the names of the two positional arguemnts from
              "fromfile" --> "install", and "example" --> "create-template"

              Added option for specifying template file instead
              of dumping the template to STDOUT

              Finally added autocomplete arg parsing to the cli tools,
              and used a unique file name in UI when creating template.

https://pagure.io/389-ds-base/issue/49791

Reviewed by: ?

- - - - -
fa41aee9 by Marc Muehlfeld at 2018-06-26T14:24:10Z
Ticket 49793 - Updated descriptions in dscreate example INF file

Description: This patch changes the format of the "dscreate example"
             output to:

                 parameter_name (type) [REQUIRED|optional]
                 Description: ...
                 Default value: ...

             Additionally, the patch updates the parameter descriptions
             to be more descriptive.

https://pagure.io/389-ds-base/issue/49793

Reviewed by: mreynolds

Signed-off-by: Mark Reynolds <mreynolds at redhat.com>

- - - - -
a2cbef23 by Viktor Ashirov at 2018-06-26T21:35:23Z
Issue 49588 - Add py3 support for tickets

Another round of py3 fixes for tests in tickets/.

https://pagure.io/389-ds-base/issue/49588

Reviewed by: mreynolds (Thanks!)

- - - - -
08b2d884 by Viktor Ashirov at 2018-06-27T18:55:57Z
Issue 49717 - Add conftest.py for tests

Bug Description:
>From the test's output it's not possible to tell what version of
389-ds-base was used during the test, what are the versions of
libraries that we depend on, etc.

Fix Description:
We can use conftest.py to print this useful information. This also will
be reflected in an html report, if py.test is used with --html option.

https://pagure.io/389-ds-base/issue/49717

Reviewed by: spichugi, amsharma, mreynolds (Thanks!)

- - - - -
6dee0c1d by Timo Aaltonen at 2018-06-29T06:33:58Z
Merge tag '389-ds-base-1.4.0.11' into master-next

- - - - -
2cab7ba3 by Timo Aaltonen at 2018-06-29T10:54:36Z
ftbs_lsoftotkn3.diff: Dropped, obsolete.

- - - - -
7d2362e2 by Timo Aaltonen at 2018-06-29T13:26:23Z
fix-linking.diff: Fix the order of linking liblslapd.la/libldaputil.la.

- - - - -
126e700f by Timo Aaltonen at 2018-06-29T13:30:57Z
Issue 49816

Fix linking order of libldaputil/libslapd

https://pagure.io/389-ds-base/issue/49816

- - - - -
fda2bd39 by Timo Aaltonen at 2018-06-29T13:46:06Z
control: Add rsync to build-depends.

- - - - -
4f9dfd58 by Timo Aaltonen at 2018-06-29T14:55:37Z
libs, dev, control: Add libsvrcore files, replace old package.

- - - - -
d053609a by Timo Aaltonen at 2018-06-29T15:00:57Z
base: Add new scripts, add python3-selinux, -semanage, -sepolicy to depends.

- - - - -
eafaf473 by Mark Reynolds at 2018-06-29T16:53:30Z
Ticket 49783 - UI - add server configuration backend

Description: This patch pulls in all the cn=config values and poplues the UI.
             it also allows you to save values to the server.

             Also cockpit can only pull in strerr messages when things fail,
             so I had to undue a recent change to redirect stderr to stdout.

https://pagure.io/389-ds-base/issue/49783

Reviewed by: spichugi & vashirov(Thanks!!)

- - - - -
0d9886e7 by Mark Reynolds at 2018-06-29T16:55:47Z
Ticket 49811 - Update man pages

Bug Description:  A recent man page scan revealed several errors and warnings.

Fix Description:  Added missing man pages, fixed syntax issues, and added
                  new man pages for config files.

                  Also added dynamic man page creation for the lib389 CLI tools
                  using python-argparse-manpages

https://pagure.io/389-ds-base/issue/49811

Reviewed by: spichugi & vashirov(Thanks!!)

- - - - -
177f5c45 by Simon Pichugin at 2018-06-29T19:01:43Z
Issue 49761 - Fix more CI test issues

Description: Mark conflict resolution test cases as 'xfail' (ds49591).
Fix SASL test. Fix Python 3 bytes issues in ds-replcheck tool.
Roll back a part of the PBKDF2_SHA256 schema fix because upgrade fails
without this part of the code.

https://pagure.io/389-ds-base/issue/49761

Reviewed by: vashirov, mreynolds (Thanks!)

- - - - -
d881a4cb by Mark Reynolds at 2018-06-30T14:28:04Z
Ticket 49791 - Update docker file for new dscreate options

Description:  Update docker file to use new option names in dscreate

ttps://pagure.io/389-ds-base/issue/49791

Review by mreynolds (one line commit rile)

- - - - -
3bdb6462 by Timo Aaltonen at 2018-07-02T08:07:46Z
dev: fix a typo

- - - - -
b903b039 by Viktor Ashirov at 2018-07-02T13:42:34Z
Issue 49820 - lib389 requires wrong python ldap library

Bug Description:
python3-ldap [1] is not the same as python-ldap [2].

[1] https://pypi.org/project/python3-ldap/
[2] https://pypi.org/project/python-ldap/

Fix Description:
Update setup.py and requirements.txt.

https://pagure.io/389-ds-base/issue/49820

Reviewed by: mreynolds (Thanks!)

- - - - -
c0a80d28 by Akshay Adhikari at 2018-07-02T14:21:38Z
Issue 49588 - Add py3 support for tickets : part-3

Description: Added py3 support by explicitly changing strings to bytes.

https://pagure.io/389-ds-base/issue/49588

Reviewed by: vashirov (Thanks!)

- - - - -
05919a64 by Timo Aaltonen at 2018-07-02T14:49:33Z
Add a package for cockpit-389-ds.

- - - - -
f80928f4 by Mark Reynolds at 2018-07-02T18:51:45Z
Ticket 49795 - UI - add "action" backend funtionality

Description:  Added the backend functionality for all the items in the "Action"
              dropdown list on the banner.

              Added ability to use the instance name in dsconf, previously
              only ldap URLs worked.

              Added more helper functions:

                popup_confirm() - this now accepts html tags so nicer output
                get_ldapurl_from_serverid(server_id) - take an instance name
                  and generate a LDAP URL from its dse.ldif

              And of course did some code cleanup.

https://pagure.io/389-ds-base/issue/49795

Reviewed by: ?

- - - - -
4404d03d by Mark Reynolds at 2018-07-02T18:54:35Z
Ticket 49811 - lib389 setup.py should install autogenerated man pages

Description:  Create and install the lib389 cli man pages in its
              setup.py file.  Previously running "make" in a dev
              environment was failing because "make' was not installing
              lib389 (and its man pages) which was breaking the build.

https://pagure.io/389-ds-base/issue/49811

Reviewed by: ?

- - - - -
ea7e9897 by Mark Reynolds at 2018-07-03T13:15:23Z
Ticket 49808 - Add option to add backend to dscreate

Description:  Added new directives to the inf file to create multiple
              backends.  Also update the UI for the new options.

              Also, changed some all the inf options to be optional
              except for the rootdn password.

              Did some other minor code cleanup

https://pagure.io/389-ds-base/issue/49808

Reviewed by: vashirov, spichugi, and firstyear (Thanks!!!)

- - - - -
90332887 by Mark Reynolds at 2018-07-05T13:36:51Z
Ticket 49813 - dscreate - add interactive installer

Description:  Added interactive installer to dscreate.  Also
              added signal handlers so that control-c does not
              cause a python stack dump in the cli tools.

              Also quieted the output for dscreate, and made
              it slightly more friendly.

https://pagure.io/389-ds-base/issue/49813

Reviewed by: spichugi(Thanks!)

- - - - -
aa3046f0 by Timo Aaltonen at 2018-07-06T06:30:36Z
rules: Clean up cruft left after build.

- - - - -
2e3b0822 by Timo Aaltonen at 2018-07-06T06:44:21Z
control: Drop dh_systemd from build-depends, bump debhelper to 11.

- - - - -
185963ef by Mark Reynolds at 2018-07-06T16:48:58Z
Ticket 49832 - remove tcmalloc references

Description:  We now bundle jemalloc, so stop building the server
              with tcmalloc.

https://pagure.io/389-ds-base/issue/49832

Reviewed by: mreynolds(one line commit rule)

- - - - -
8fa838a4 by Mark Reynolds at 2018-07-09T15:47:29Z
Ticket 49830 - Import fails if backend name is "default"

Bug Description:  The server was previously reserving the backend
                  name "default".  If you tried to import on a
                  backend with this name the import would skip all
                  child entries

Fix Description:  Change the default backend name to something
                  obscure, instead of "default".

                  Also improved lib389's dbgen to generate the
                  correct "dc" attribute value in the root node.

https://pagure.io/389-ds-base/issue/49830

Reviewed by: spichugi(Thanks!)

- - - - -
5537467e by Ludwig Krispenz at 2018-07-10T13:04:20Z
Ticket 49780 - acl_copyEval_context double free

Bug: There is a connection an operation extension to keep evaluation
     results cached across multiple operations.
     In the geteffective for each evaluation a sepatate temporary connection
     extension is created and freed. If there are several concurrent operations
     in one connection the operation extension destructor can try to access
     an already freed connection extension.

Fix: Do not free the connectcion extension during the life time of a
     connection. If a temporary extension should be used, lock the connection extension,
     copy the content  to a saved structure and copy the temporary data to the
     connection extension. When done with the temporary extension reverse this.
     To make this work the locks need to be directly created, the use of the limited
     preallocated array of locks could lead to a situation where both the main and
     the temporary extension would get the sam elock assigned.

Reviewed by: Thierry and Mark, Thanks

- - - - -
4206d278 by Thierry Bordaz at 2018-07-10T13:30:48Z
Ticket 48818 - For a replica bindDNGroup, should be fetched the first time it is used not when the replica is started

Bug Description:
	The fetching of the bindDNGroup is working as designed but this ticket is to make it more flexible

	At startup, if the group does not contain the replica_mgr.
	No replication session will succeed until bindDnGroupCheckInterval delay.
	updatedn_group_last_check is the timestamp of the last fetch. At startup
	updatedn_group_last_check is set to the current time. So the next fetch will happen not before
	updatedn_group_last_check+bindDnGroupCheckInterval.

	If the groupDn is changed after startup, no incoming replication can happen for the first
	bindDnGroupCheckInterval seconds

Fix Description:
	The fix consist to unset updatedn_group_last_check so that the group will be fetch when the first
	incoming replication session will happen.

https://pagure.io/389-ds-base/issue/49818

Reviewed by: Mark Reynolds, Simon Spichugi (thanks !!!)

Platforms tested: F27

Flag Day: no

Doc impact: no

- - - - -
df95edbe by Stanislav Laznicka at 2018-07-10T14:40:34Z
Issue 49835 - lib389: fix logging

The python standard logging module allows to postpone the string
interpolation in logs till the time the interpolation is actually
needed (note that you may be running in a certain logging level
that won't need to display/write all strings). This behavior is
there to comply with the logging requirement so that logs don't
affect the performance if no logging happens.

The above behavior only works when strings are passed as arguments
to the logging methods. If the interpolation is invoked at place
either by using the `%` notation or by using the `str.format()`
method, python logging can't perform its performance heuristics
since the strings get interpolated before they are handed to the
logging methods.

This commit fixes the improper behavior observed in the lib389
library, plus adds some fixes to improper invokes of the logging
methods - multiple arguments if formatting string does not contain
formatting variables -  which would cause internal logging errors
that don't cause the scripts failures, which is why they probably
weren't reported yet.

https://pagure.io/389-ds-base/issue/49835

Reviewed by: mreynolds, spichugi

Signed-off-by: Simon Pichugin <simon.pichugin at gmail.com>

- - - - -
2fa04088 by Simon Pichugin at 2018-07-10T15:18:00Z
Issue 49640 - Cleanup plugin bootstrap logging

Bug Description: We add PBKDF2_SHA256 password storage schema two times. During:
1. the dse.ldif parsing;
2. the bootstrap plugin operation.
It causes the error to appear during the startup.

Fix Description: Make plugin_setup() function report the error to TRACE log level
if the plugin already exists. We will report the error in ERR log level during
the config bootstrap anyway (code path for the 1st option from bug description).
For 2nd option, report the error to TRACE if it is 'already exist' issue
and to ERR if it is any other case.

Make the plugin_setup returns more consistent.

https://pagure.io/389-ds-base/issue/49640

Reviewed by: mreynolds, mhonek (Thanks!)

- - - - -
60cb5204 by Mark Reynolds at 2018-07-12T14:25:41Z
Ticket 49840 - ds-replcheck command returns traceback errors against ldif files having garbage content when run in offline mode

Description:  Added a basic check to see if the LDIF files are actually
              LDIF files.  Also added checks that the database RUV are
              present as well.

https://pagure.io/389-ds-base/issue/49840

Reviewed by: spichugi(Thanks!)

- - - - -
6d4caac0 by Mark Reynolds at 2018-07-13T12:27:43Z
Ticket 49546 - Fix issues with MIB file

Description:  Change dsMaxThreadsHit to dsMaxThreadsHits, and set the
              proper object type for dsIntIndex

https://pagure.io/389-ds-base/issue/49546

Reviewed by: spichugi & firstyear(Thanks!!)

- - - - -
fbe62eb7 by Amita Sharma at 2018-07-16T13:46:40Z
Test that memberof does not fail if group is moved into scope

Description: Added a test test_memberof_group() to test bug1551071,
it tests that memberof does not fail if group is moved into scope

https://pagure.io/389-ds-base/issue/49161

Reviewed by: Simon

- - - - -
7892486b by Stanislav Laznicka at 2018-07-16T18:57:50Z
Issue 49844 - lib389: don't set up logging at module scope

Bug description: lib389 was calling `logging.basicConfig()`
at several places at module scope level. This was causing
imports from these modules to add an unwanted handler
to the root logger of the python's standard logging module.

Fix description: Set up logging only in the scripts that are
using lib389.
ALso, remove unused imports in __init__.py

https://pagure.io/389-ds-base/issue/49844

Reviewed by: spichugi, mreynolds (Thanks!)

Signed-off-by: Simon Pichugin <spichugi at redhat.com>

- - - - -
e81fa859 by Thierry Bordaz at 2018-07-17T09:34:26Z
Ticket 49789 - By default, do not manage unhashed password

Bug Description:
    By default, unhashed#user#password is recorded into changelog database.
    It is a specific use when some plugin need to know the clear text password on update.
    This should be disabled ('off') by default

Fix Description:
    Switch the default value from 'on' to 'off'

https://pagure.io/389-ds-base/issue/49789

Reviewed by: Viktor Ashirov, Simon Pichugi, Mark Reynolds

Platforms tested: F27

Flag Day: no

Doc impact: no

- - - - -
4f5f6bb5 by Mark Reynolds at 2018-07-17T14:23:31Z
Ticket 49813 - Revised interactive installer

Description:
              Removed some advanced settings from the install questions.
              Moved the signal handlers to non-verbose runs.
              Fixed some mixed case issues.
              Added option for sample entries.
              Added "interactive" argument, and restored "fromfile"
              from "install".

https://pagure.io/389-ds-base/issue/49813

Reviewed by: mmuehlfeldrh & spichugi(Thanks!!)

- - - - -
9110d2b4 by Mark Reynolds at 2018-07-17T15:48:05Z
Ticket 48377 - Move jemalloc license to /usr/share/licences

Description:  Move jemalloc license (COPYING.jemalloc) to /usr/share/licences

              also added nss version dependancy that was missing

https://pagure.io/389-ds-base/issue/48377

Reviewed by: mreynolds(one line commit rule)

- - - - -
3fc43bdc by Mark Reynolds at 2018-07-17T17:34:03Z
Bump version to 1.4.0.12

- - - - -
a9fa2104 by Mark Reynolds at 2018-07-18T13:31:42Z
Ticket 49857 - RPM scriptlet for 389-ds-base-legacy-tools throws an error

Bug Description:  If no instances are present when the legacy tool
                  subpackage is installed the RPM scriplet throws an
                  error:

                       line 29: [: -eq: unary operator expected

Fix Description:  The error occurs because a variable that counts the
                  existing instances is not initialized.

https://pagure.io/389-ds-base/issue/49857

Reviewed by: mreynolds (one line commit rule)

- - - - -
172c60a2 by Mark Reynolds at 2018-07-18T16:01:53Z
Ticket 49789 - backout original fix as it caused a regression in FreeIPA

Description:  This change broke FreeIPA, so for now we need to back it out

https://pagure.io/389-ds-base/issue/49789

- - - - -
ba5900ee by Mark Reynolds at 2018-07-18T17:32:47Z
Ticket 49806 - Add SASL functionality to CLI/UI

Description:  Add SASL functionality to dsconf and UI

              Improved installer to load the template-sasl ldif
              file.

              Also cleaned up some of the "clear form" functions
              to be more efficient using jquery.

https://pagure.io/389-ds-base/issue/49806

Reviewed by: spichugi(Thanks!)

- - - - -
dd183895 by Mark Reynolds at 2018-07-19T17:02:16Z
Ticket 49854 - ns-slapd should create run_dir and lock_dir directories at startup

Description:  dscreate was not creating its config file in /etc/tmpfiles.d/ like
              setup-ds.pl used to do.  The absence of this config file prevented
              the server from being started after a reboot.

https://pagure.io/389-ds-base/issue/49854

Reviewed by: vashirov(Thanks!)

- - - - -
345221c0 by Mark Reynolds at 2018-07-19T18:38:15Z
Bump version to 1.4.0.13

- - - - -
52b89974 by Mark Reynolds at 2018-07-23T21:40:15Z
Ticket 49867 - Fix CLI tools' double output

Description:  This is a regression from a previous fix.  Do not call
              logging.basicConfig() in cli tools

https://pagure.io/389-ds-base/issue/49867

Reviewed by: spichugi(Thanks!)

- - - - -
fd8ca9aa by Mark Reynolds at 2018-07-24T14:51:25Z
Ticket 49794 - Add pam_pwquality features to password syntax checking

Description:  Added the following pam_pwquality fartures to DS.  We can
              not add all the features because some of them require that
              you have the the previous or current password in clear text.

               New features:

                 - Check password is not in dictionary
                 - Password is not a palindrome
                 - Maximum allowed monotonic sequence characters
                 - Maximum allowed monotonic sequence characters that can
                   be repeated.
                 - Maximum number of consectuve characters from the same
                   class of characters (digits, alphas, specials, etc)
                 - List of words that are not allowed to appear in the new
                   password
                 - List of attributes to check in the user's entry to see
                   if those values are in the new password.

https://pagure.io/389-ds-base/issue/49794

Reviewed by: vashirov & spichugi(Thanks!!)

- - - - -
7e49dec0 by Mark Reynolds at 2018-07-25T22:16:10Z
Ticket 49837 - Add new password policy attributes to UI

Description:  Added new password policy features to UI.

              Also made change to instance creation to line up
              with changes going on in lib389

https://pagure.io/389-ds-base/issue/49837

Reviewed by: spichugi(Thanks!)

- - - - -
c439b920 by Simon Pichugin at 2018-07-27T14:10:38Z
Issue 49381 - Refactor the plugin test suite docstrings

Description: Remove attr_uniqueness_test.py and dna_test.py
because they are present in acceptance_test.py.
Refactor the docstrings in the existing suites.

https://pagure.io/389-ds-base/issue/49381

Reviewed by: vashirov, mreynolds (Thanks!)

- - - - -
7b3c401a by Simon Pichugin at 2018-07-30T12:22:23Z
Issue 49761 - Fix replication test suite issues

Description - the issues:
cleanallruv - add 'certify' to the abort tasks certify and adjust a timeout;
encryption_cl5 - add the encryption attributes to the schema
(we had used extensibleObject before but now we use nsChangelogConfig);
tombstone - fix how Tombstone(DSLdapObject) handles the searches.

https://pagure.io/389-ds-base/issue/49761

Reviewed by: vashirov, mreynolds (Thanks!)

- - - - -
b14c836b by Amita Sharma at 2018-07-31T12:05:11Z
Issue 48056 - Add more test cases to the basic suite

Description: Added a test for anonymous search with various filters
Added a test to verify bug915801
Added a test to verify bug192901
Added a test to verify bug1044135
Added markers for password and filter test cases

https://pagure.io/389-ds-base/issue/48056

Reviewed by: Simon and Viktor

- - - - -
ee5c533d by Timo Aaltonen at 2018-07-31T14:35:56Z
Add varions libjs packages to cockpit-389-ds Depends, add the rest to d/missing-sources.

- - - - -
557a9a1d by Timo Aaltonen at 2018-07-31T14:41:05Z
control, rules: migrate to dh 11

- - - - -
ab2bf812 by Timo Aaltonen at 2018-07-31T14:59:48Z
add more unminified js

- - - - -
a00aeb27 by Timo Aaltonen at 2018-07-31T15:48:06Z
copyright: Updated. (Closes: #904760)

- - - - -
a853ead3 by Timo Aaltonen at 2018-07-31T16:03:34Z
changelog, fix a typo

- - - - -
092b62a4 by Timo Aaltonen at 2018-07-31T16:08:18Z
fix copyright

- - - - -
b7fe3ea9 by Timo Aaltonen at 2018-07-31T17:23:02Z
Merge tag '389-ds-base-1.4.0.13' into master-next

- - - - -
f2254733 by Timo Aaltonen at 2018-07-31T17:45:12Z
update changelog

- - - - -
ee619c25 by Timo Aaltonen at 2018-07-31T17:46:13Z
Revert "control: Drop 389-ds metapackage, the other 389-ds-* packages are EOL'd upsream and will be removed soon."

This reverts commit bffa315b777248c5ec6f627478ebfbb372458eb0.

- - - - -
c1c67ef6 by Timo Aaltonen at 2018-07-31T18:52:20Z
control: Modify 389-ds to depend on cockpit-389-ds and drop the old GUI packages which are deprecated upstream.

- - - - -
a9b1666d by Timo Aaltonen at 2018-07-31T19:30:52Z
drop fix-linking.diff, upstream

- - - - -
8f634923 by Timo Aaltonen at 2018-07-31T20:16:50Z
dont-build-new-manpages.diff: Debian doesn't have argparse-manpage, so in order to not FTBFS don't build new manpages.

- - - - -
028dadcd by Timo Aaltonen at 2018-07-31T20:35:17Z
base.install: Add man5/*.

- - - - -
222d75b7 by Timo Aaltonen at 2018-07-31T20:46:28Z
releasing package 389-ds-base version 1.4.0.13-1

- - - - -
ba8f57d2 by William Brown at 2018-08-03T01:10:59Z
49881 - Missing check for crack.h

A check for crack.h is missing from configure.ac. Add it.

https://pagure.io/389-ds-base/issue/49881

Author: William Brown <william at blackhats.net.au>

- - - - -
74498103 by Mark Reynolds at 2018-08-06T17:53:34Z
Ticket 49866 - Add password policy features to CLI/UI

Description:  Add global and local password policy functionality to
              dsconf and UI.

https://pagure.io/389-ds-base/issue/49866

Reviewed by: ?

- - - - -
0d1eded7 by Mark Reynolds at 2018-08-07T12:43:11Z
Ticket 49888 - Use perl filter in rpm specfile

Description:  Apply perl filter to legacy tools subpackage to prevent 389
              from "Providing" its perl modules.

https://pagure.io/389-ds-base/issue/49888

Reviewed by: vashirov(Thanks!)

- - - - -
e4831bc4 by William Brown at 2018-08-07T23:19:19Z
49884 - Improve nunc-stans test to detect socket errors sooner

While testing on a fresh machine (without ipv6) I noticed the ns test
would fail. This led me to improve the state of the ns stress test
code to remove the legacy atomic, and check assertions of the sockets.

https://pagure.io/389-ds-base/pull-request/49884

Author: William Brown <william at blackhats.net.au>

- - - - -
ad640e96 by Viktor Ashirov at 2018-08-08T12:12:40Z
Issue 48377 - Update file name for LD_PRELOAD

Bug Description:
We ship versioned libjemalloc.so.2, but LD_PRELOAD still uses
unversioned file name.

Fix Description:
Update LD_PRELOAD to use versioned .so name.

https://pagure.io/389-ds-base/issue/48377

Reviewed by: mreynolds (Thanks!)

- - - - -
2f2d3b1d by Mark Reynolds at 2018-08-08T21:19:27Z
Ticket 49893 - disable nunc-stans by default

Description:  Until nunc-stans is stablized we need to disable it

https://pagure.io/389-ds-base/issue/49893

Reviewed by: ?

- - - - -
90bf1799 by Mark Reynolds at 2018-08-09T19:11:33Z
Ticket 49029 - improve internal operations logging

Description:  This patch uses "local thread storage" to keep track of
              the connection and op id numbers for internal operations.
              This makes it easier to follow the access logs and know
              what connection is actually doing what.

              It also maintains its on operation id count for each
              internal operation.  This adds complexity once internal ops
              start nesting (plugins calling plugins).  To handle this
              two LTS variables were added to keep track of the nested
              level or count, and one to know once a series of nested
              operations are finally unnested/complete.  This is needed
              to maintain the correct internal operation id counter correct
              and in sequence.

              Also organized the local thread storage initialization into
              a single function - there was no need to have separate
              functions.

              Thanks lkrispenz for improving the logging format, and
              providing the initial patch this was all based off of.

https://pagure.io/389-ds-base/issue/49029

Reviewed by: tbordaz & lkrispen(Thanks!)

- - - - -
c989e18f by Mark Reynolds at 2018-08-09T19:38:57Z
Ticket 49890 : ldapsearch with server side sort crashes the
 ldap server

Bug Description:
        Server side sort with a specified matching rule trigger a crash

Fix Description:
        Check if the we are able to index the provided value.
        If we are not then slapd_qsort returns an error (LDAP_OPERATION_ERROR)

https://pagure.io/389-ds-base/issue/49890

Reviewed by: mreynolds

Platforms tested: F27

Flag Day: no

Doc impact: no

- - - - -
b413ba02 by Mark Reynolds at 2018-08-10T13:39:05Z
Ticket 49891 - Use "--python3" macro for python scripts

Description:  Make sure all python scripts use the macro "__python3"
              for its shebang.

https://pagure.io/389-ds-base/issue/49891

Reviewed by: vashirov(Thanks!)

- - - - -
520e2f9c by Mark Reynolds at 2018-08-10T15:21:17Z
Ticket 49891 - Use "--python3" macro for python scripts (remove readnsstate)

Description:  The spec file can not update readnsstate script, so skip it
              from the last commit

- - - - -
bd3daf13 by Mark Reynolds at 2018-08-10T15:22:45Z
Bump version to 1.4.0.14

- - - - -
9ac6112b by William Brown at 2018-08-12T23:41:31Z
49885 - On some platform fips does not exist

Fix test detection to not fail with exceptions if fips proc files
are not present

https://pagure.io/389-ds-base/issue/49885

Author: William Brown <william at blackhats.net.au>

- - - - -
11d689da by Mark Reynolds at 2018-08-14T13:13:58Z
Ticket 48377 - Only ship libjemalloc.so.2

Description:  We are only supposed to ship libjemalloc.so.2, and nothing else.

https://pagure.io/389-ds-base/issue/48377

Reviewed by: vashirov(Thanks!)

- - - - -
160e0273 by Amita Sharma at 2018-08-15T13:28:39Z
Issue 48061 : CI test - config

Description: Add more test cases to the config suite

https://pagure.io/389-ds-base/issue/48061

Reviewed by: Simon

- - - - -
9f9f4a9d by Mark Reynolds at 2018-08-16T14:18:44Z
Ticket 49029 - Internal logging thread data needs to allocate int pointers

Bug Description:

The original version of the fix incorrectly used stack pointers
to update the thread data.  These pointers would go out of scope
and could cause a crash when updated.

Fix Description:

Allocate the integer pointers at initalization time for each
worker thread including main() thread.  Also cleaned up other
areas of this code/feature.

Passed ASAN tests.

https://pagure.io/389-ds-base/issue/49029

Reviewed by: vashirov & lkrispen(Thanks!!)

- - - - -
94f30daf by Mark Reynolds at 2018-08-16T17:01:46Z
Bump version to 1.4.0.15

- - - - -
17dc2b07 by Timo Aaltonen at 2018-08-22T21:25:18Z
Merge branch 'upstream'

- - - - -
31996474 by Timo Aaltonen at 2018-08-22T21:28:20Z
update the changelog

- - - - -
8a7a4f0b by Timo Aaltonen at 2018-08-22T21:37:31Z
control: Add libcrack2-dev to build-depends.

- - - - -
54b166ba by Timo Aaltonen at 2018-08-22T21:46:55Z
releasing package 389-ds-base version 1.4.0.15-1

- - - - -


21 changed files:

- Makefile.am
- − README
- + README.md
- VERSION.sh
- configure.ac
- debian/389-ds-base-dev.install
- debian/389-ds-base-libs.install
- debian/389-ds-base.install
- debian/changelog
- + debian/cockpit-389-ds.install
- debian/compat
- debian/control
- debian/copyright
- + debian/missing-sources/bootpopup.js
- + debian/missing-sources/bootstrap.js
- + debian/missing-sources/c3.js
- + debian/missing-sources/d3.js
- + debian/missing-sources/jquery-1.12.4.js
- + debian/missing-sources/jquery-3.3.1.js
- + debian/missing-sources/jquery-ui.js
- + debian/missing-sources/jquery.dataTables.js


The diff was not included because it is too large.


View it on GitLab: https://salsa.debian.org/freeipa-team/389-ds-base/compare/11465a5bd33c23a72faa857a5cdaece527d0621c...54b166ba6709e3b765dee4d326f62092e3f41630

-- 
View it on GitLab: https://salsa.debian.org/freeipa-team/389-ds-base/compare/11465a5bd33c23a72faa857a5cdaece527d0621c...54b166ba6709e3b765dee4d326f62092e3f41630
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-freeipa-devel/attachments/20180822/b8b39421/attachment-0001.html>


More information about the Pkg-freeipa-devel mailing list