Business Unit Count doesn't match Scheduled Task Export

Question

I have a scheduled task that exports the members from a Business unit. However the number of members in the Business Unit (4648), does not equal the number of rows exported (4738). I can identify some of the extra records, but when I search for them in the Adaxes Console using the LDAP search, I don't find them, so I have no idea where these are being picked up from. I don't have any extra processing going on in my scheduled task that could account for the retrieving of records outside of what's contained in the Business Unit.

If someone could point me in the direction of where else I could look to determine the discrepancy, I would greatly appreciate it.

Comments

I've found the culprit! The extra accounts being exported are showing up as Unmanaged User Accounts. They don't display in the Business Unit listing, so in my mind this is an Adaxes bug. Please confirm.

---

Hello,

That looks like a bug. Can you post here or send us to support[at]adaxes.com the following to help us troubleshooting the issue:

1. A screenshot with the number of users in the Business Unit.
2. The script that exports the Business Unit members.
3. A screenshot of the Membership Rules of your Business Unit.

---

It was determined that we were one version behind so I just installed the latest Admin. Console (v 3.7.13122.0) and there's still an issue. Here's the requested information.

1. Business Unit Screen Shot:

2. Export Script:

Import-Module Adaxes

# This exports multiple attributes and creates a CSV file minus the Header row.
# Also retrieves the common name for the positionPrimarySupervisor.
# Formats the user name as lastname , firstname middleInitial where possible, otherwise lastname , firstname

$sortBy = "employeeID" # sort search results by UID
$sortDirection = "Ascending" # sort in ascending order
$envir = "1"    #Unrem for dev/test
#$envir = "2"    #Unrem for production environment, and rem the line above
$businessUnitName = "For Export - Active Users (New)"

try
{
    # Read the txt file that has the file export location and field information and then export data
    $Context.LogMessage("About to export data to LD Service Desk","Information")
    if ($envir -eq "1"){    #if dev/test environment
            $filePath = "c:\AdaxesConfigurationFiles\(DEV)DirInt0002_Exp_Info.csv"   #Read file from Adaxes c: drive that contain export information
        } else {                #if production environment
            $filePath = "c:\AdaxesConfigurationFiles\DirInt0002_Exp_Info.csv"   #Read file from Adaxes c: drive that contain export information
        }

    # Read the file and put into array variables
    $ExpPath = @()
    $LDAPNames = @()
    $CSVNames = @()

    Import-Csv $filePath |`
        ForEach-Object {
            $ExpPath += $_."ExportPath"
            $LDAPNames += $_."LDAPName"
            $CSVNames += $_."FieldsOut"
        }

    if($ExpPath.Length -eq 0)
    {
        $Context.LogMessage("No export path found.  Processing terminated.", "Warning")
        return
    }

    $Context.LogMessage("Export path is: " + $ExpPath, "Information")
    $Context.LogMessage("LDAP Names are: " + $LDAPNames, "Information")
    $Context.LogMessage("Output Fields: " + $CSVNames, "Information")

    # Parse fields

    $eachFieldIn = @($LDAPNames[0].split(","))
    $eachFieldOut = @($CSVNames[0].split(","))

    if (($eachFieldIn.Length -eq 0) -or ($eachFieldOut.Length -eq 0))
    {
       $Context.LogMessage("Fields missing during import.  Processing terminated.", "Warning")
        return 
    }

    if (($eachFieldIn.Length) -ne ($eachFieldOut.Length))
    {
       $Context.LogMessage("Field in/out count mismatch during import.  Processing terminated.", "Warning")
        return 
    }

    # Build path for the user report file
    $exportFile = $ExpPath[0]

    # Delete the existing file if it already exists
    if (Test-Path $exportFile) 
    {
        Remove-Item $exportFile
    }

    # Create new file for the user report
    New-Item $exportFile -Type File

    # Find the Business Unit
    $businessUnitsPath = $Context.GetWellKnownContainerPath("BusinessUnits")
    $usersearcher = $Context.BindToObject($businessUnitsPath)
    $usersearcher.SearchFilter = "(&(objectCategory=adm-BusinessUnit)(name=$businessUnitName))"
    $usersearcher.PageSize = 500
    $usersearcher.SearchScope = "ADS_SCOPE_SUBTREE"
<#
    $userSearcher.SetPropertiesToLoad($eachFieldIn)

    $sortOption = New-Object "Softerra.Adaxes.Adsi.AdmSortOption"
    $sortOption.PropertyName = $sortBy
    $sortOption.Direction = $sortDirection
    $userSearcher.Sort = $sortOption
#>    
    # Get the user information from the search results add them to the file
    try
    {
        $userResult = $userSearcher.ExecuteSearch()
        $objects = $userResult.FetchAll()

        if ($objects.Length -gt 1)
        {
            $Context.LogMessage("Found more than one Business Unit with name '$businessUnitName'.", "Warning")
            return
        }
        if ($objects.Length -eq 0)
        {
            $Context.LogMessage("Business Unit '$businessUnitName' does not exist.", "Error")
            return
        }

        # Get the Business Unit Members
        $unit = $Context.BindToObject($objects[0].AdsPath)
        $members = $unit.Members()

        $totalUserCount = $members.Count
        $Context.LogMessage("The total number of users is: " + $totalUserCount, "Information")

        $count = 0
        $report = @()

        for ($i = 0; $i -lt $totalUserCount; $i++)
        {

                $userRec = $members.GetObject($i)
                $reportRecord = New-Object PSObject
                foreach ($propertyName in $eachFieldIn)
                {
                    try
                    {
                        switch ($propertyName)
                        {
                            "positionPrimarySupervisor" {
                                try {
                                   $managerDN = $userRec.Get($propertyName)
                                   try
                                      {
                                          $supervisor = $Context.BindToObjectByDn($managerDN)
                                          $value = $supervisor.Get("cn")
                                      }
                                      catch
                                      {
                                          $value = ""
                                      }
                                }
                                catch {
                                   $value = "" 
                                }
                            }

                            "cn" {
                                # format name as lastname , firstname middleinitial if possible, otherwise as lastname , firstname
                                $lname = $userRec.Get("sn")
                                try {
                                    $fname = $userRec.Get("preferredFirstName")
                                }
                                catch {
                                    $fname = $userRec.Get("givenName")
                                }   

                                try {
                                   $mname = $userRec.Get("middleName") 
                                }
                                catch {
                                   $mname = "" 
                                }

                                $value = $lname + " , " + $fname + " " + $mname
                            }

                            Default {
                                $value = $userRec.Get($propertyName)
                            }

                        }  # end switch $propertyName

                    }
                    catch
                    {
                        $value = "" # The property is empty
                    }

                    $reportRecord | Add-Member -Name $propertyName -Value $value -MemberType NoteProperty
                } # end foreach propertyname
                $report += $reportRecord

                $count++
        }  # end foreach user
    }
    finally
    {
        $userResult.Dispose()
    }

    #$report | Export-Csv $exportFile -NoTypeInformation  *** Uncomment this line to export with the Header row
    $report | ConvertTo-Csv -NoTypeInformation | select -Skip 1 | Set-Content $exportFile

    $Context.LogMessage("Total records exported = " + $count, "Information")
}
catch  #catch any unresolved errors
{
    $ErrorMessage = $_.Exception.Message
    [string]$LineNumber = $_.InvocationInfo.ScriptLineNumber
    [string]$Offset = $_.InvocationInfo.OffsetInLine
    [string]$errLine = $_.InvocationInfo.Line

    Write-Error $ErrorMessage
    $Context.LogMessage("At Line #: " + $LineNumber + " at char " + $Offset, "Error")
    $Context.LogMessage("Executing: " + $errLine, "Error")  
}

ExpandCopy code

3. Membership Rules:
   (&(objectCategory=person)(objectClass=user)(!(employeeID=000000))(!(userAccountControl:1.2.840.113556.1.4.803:=2))(!(employeeID=NMU))(!(employeeID=?))(employeeID=*))

4. Although not requested, here's the activity log from the Export scheduled task:

About to export data to LD Service Desk
Export path is: \\data1\isshared\LD ServiceDesk\TEST_LAndesk HR Data Export.csv
LDAP Names are: cn,employeeID,telephoneNumber,positionPrimarySupervisor,positionSupervisorUID,mail,department,division,buildingName,locationFloor,roomNumber,terminationDate
Output Fields: Name,UID,Phone,Manager,Managers UID,Email Address,Dept,Division,Building,Floor,Room,termDate
The total number of users is: 4721
Total records exported = 4721

---

Do you have a single instance of Adaxes Service or multiple instances installed on multiple computers? If you have multiple instances, can you re-check the number of the Business Unit members on all the services?

---

Yes, we have two instances. I just checked and the member count is identical.

---

Hello,

We've checked your script with your Business Unit Membership Rules in our environment, but we didn't observe such an issue. In our environment, the number of users is identical in both cases. For further troubleshooting, can you send us a backup of your Adaxes Service? For information on how to make the backup, see the following help article: http://www.adaxes.com/help/?HowDoI.Mana ... ation.html.

---

Backup configuration file attached...

---

Hello,

The thing is that you included 2 domains in the Activity Scope of the Scheduled Task that sets Unmanaged Accounts. Because of this, the script used in the task runs 2 times within a short period of time. Since Adaxes needs some time to rebuild the list of Unmanaged Accounts, running the script for the 2nd time interrupts the process and can cause the issue with calculating the number of managed accounts.

To remedy the issue, you need to remove one of your domains from the the Activity Scope of the task. It does not matter which one. By the design of the script, a domain specified in the Activity Scope only triggers execution of the script. The Activity Scope does not affect the scope of AD users added to Unmanaged Accounts. The scope of managed / unmanaged accounts is specified directly in the script.

To do this:

1. In the Activity Scope section of the Scheduled Task properties, right-click any of the domains and select Remove.
   
2. Click Save changes.
   
3. Before re-checking the issue with the Business Unit, wait for the Scheduled Task to run based on the schedule or run it manually.

---

The scheduled task that uses the Business Unit only has one Activity Scope Assigned Over domain selected. I believe you were looking at the original Scheduled Task created before using the Business Unit. See attached screen capture:

---

Hello,

It seems like we've found the cause for the issue. You have 2 Scheduled Tasks that have almost the same functionality, but have a very important difference between them: DirInt0002_Exp_LD_SD and DirInt0002_Exp_LD_SD(BU). Both the tasks build a report, the format is the same, both the tasks use the same LDAP filter to search for users, and, which is more important, both the task save results to the same CSV file.

The difference is that the DirInt0002_Exp_LD_SD(BU) task creates reports on the basis of members of a Business Unit, which means that it will include only managed user accounts. The DirInt0002_Exp_LD_SD task uses a LDAP search to build reports, and the search is configured so that both managed and unmanaged accounts are returned.

It looks like the 2nd task simply overwrites the CSV file created by the 1st task, hence the difference. To check this, try disabling the DirInt0002_Exp_LD_SD Scheduled Task. Then, tun the DirInt0002_Exp_LD_SD(BU) task again.

---

I am not running both of the scheduled tasks. The one without the '(BU)' at the end is the initial scheduled task created that is no longer being used because it wasn't able to look at two different OUs and compile the result in one CSV. I'm ONLY running the one with '(BU)' at the end and not on a schedule yet since we're still in the Development stage of the project. The 'conflict' scheduled task hasn't run since July 25th, 2016, so this cannot be the issue. Please tell me what's been done to recreate my issue. Did you 'import' our service image onto a VM and actually run the scheduled task? Is there some other way that we can resolve this via a Remote session into our environment? It feels like we're just spinning our wheels!!!

Answer 1

We did import your configuration, and we did test your Scheduled Tasks and Business Unit. The issue didn't get reproduced with the DirInt0002_Exp_LD_SD(BU) Scheduled Task until today. It looks like in certain conditions, the IAdmBusinessUnit::Members() method can indeed return Unmanaged Accounts. We are sorry for the inconvenience.

We've forwarded the issue to our engineers. We'll update this topic as soon as we find a solution to it.

Comments

Hello,

The IAdmBusinessUnit::Members() method returns unmanaged user accounts if there are more than 1000 unmanaged user accounts. We'll fix the issue in the nearest minor release. Until the issue is fixed, you can use the following version of the script that uses the IAdmBusinessUnit::GetMemberGuids() to get Business Units. In this version of the script, the issue is not reproduced.

Thank you very much for the bugreport!

The script:

Import-Module Adaxes

# This exports multiple attributes and creates a CSV file minus the Header row.
# Also retrieves the common name for the positionPrimarySupervisor.
# Formats the user name as lastname , firstname middleInitial where possible, otherwise lastname , firstname

$sortBy = "employeeID" # sort search results by UID
$sortDirection = "Ascending" # sort in ascending order
$envir = "1"    #Unrem for dev/test
#$envir = "2"    #Unrem for production environment, and rem the line above
$businessUnitName = "For Export - Active Users (New)"

try
{
    # Read the txt file that has the file export location and field information and then export data
    $Context.LogMessage("About to export data to LD Service Desk","Information")
    if ($envir -eq "1"){    #if dev/test environment
            $filePath = "c:\AdaxesConfigurationFiles\(DEV)DirInt0002_Exp_Info.csv"   #Read file from Adaxes c: drive that contain export information
        } else {                #if production environment
            $filePath = "c:\AdaxesConfigurationFiles\DirInt0002_Exp_Info.csv"   #Read file from Adaxes c: drive that contain export information
        }

    # Read the file and put into array variables
    $ExpPath = @()
    $LDAPNames = @()
    $CSVNames = @()

    Import-Csv $filePath |`
        ForEach-Object {
            $ExpPath += $_."ExportPath"
            $LDAPNames += $_."LDAPName"
            $CSVNames += $_."FieldsOut"
        }

    if($ExpPath.Length -eq 0)
    {
        $Context.LogMessage("No export path found.  Processing terminated.", "Warning")
        return
    }

    $Context.LogMessage("Export path is: " + $ExpPath, "Information")
    $Context.LogMessage("LDAP Names are: " + $LDAPNames, "Information")
    $Context.LogMessage("Output Fields: " + $CSVNames, "Information")

    # Parse fields

    $eachFieldIn = @($LDAPNames[0].split(","))
    $eachFieldOut = @($CSVNames[0].split(","))

    if (($eachFieldIn.Length -eq 0) -or ($eachFieldOut.Length -eq 0))
    {
       $Context.LogMessage("Fields missing during import.  Processing terminated.", "Warning")
        return 
    }

    if (($eachFieldIn.Length) -ne ($eachFieldOut.Length))
    {
       $Context.LogMessage("Field in/out count mismatch during import.  Processing terminated.", "Warning")
        return 
    }

    # Build path for the user report file
    $exportFile = $ExpPath[0]

    # Delete the existing file if it already exists
    if (Test-Path $exportFile) 
    {
        Remove-Item $exportFile
    }

    # Create new file for the user report
    New-Item $exportFile -Type File

    # Find the Business Unit
    $businessUnitsPath = $Context.GetWellKnownContainerPath("BusinessUnits")
    $usersearcher = $Context.BindToObject($businessUnitsPath)
    $usersearcher.SearchFilter = "(&(objectCategory=adm-BusinessUnit)(name=$businessUnitName))"
    $usersearcher.PageSize = 500
    $usersearcher.SearchScope = "ADS_SCOPE_SUBTREE"
<#
    $userSearcher.SetPropertiesToLoad($eachFieldIn)

    $sortOption = New-Object "Softerra.Adaxes.Adsi.AdmSortOption"
    $sortOption.PropertyName = $sortBy
    $sortOption.Direction = $sortDirection
    $userSearcher.Sort = $sortOption
#>    
    # Get the user information from the search results add them to the file
    try
    {
        $userResult = $userSearcher.ExecuteSearch()
        $objects = $userResult.FetchAll()

        if ($objects.Length -gt 1)
        {
            $Context.LogMessage("Found more than one Business Unit with name '$businessUnitName'.", "Warning")
            return
        }
        if ($objects.Length -eq 0)
        {
            $Context.LogMessage("Business Unit '$businessUnitName' does not exist.", "Error")
            return
        }

        # Get the Business Unit Members
        $unit = $Context.BindToObject($objects[0].AdsPath)
        $members = $unit.Members()

        $totalUserCount = $members.Count

        #$Context.LogMessage("The total number of users is: " + $count, "Information")

        $count = 0
        $report = @()

        $configurationSetSettingsPath = $Context.GetWellKnownContainerPath("ConfigurationSetSettings")
        $admConfigurationSetSettings = $Context.BindToObject($configurationSetSettingsPath)

        for ($i = 0; $i -lt $totalUserCount; $i++)
        {
                # Check whether the user is managed by Adaxes
                $userRec = $members.GetObject($i)
                $userSidsBytes = $userRec.Get("ObjectSid")
                $sid = New-Object "Softerra.Adaxes.Adsi.Sid" @($userSidsBytes, 0)
                if ($admConfigurationSetSettings.IsUnmanagedAccount($sid))
                {
                    continue
                }

                $reportRecord = New-Object PSObject
                foreach ($propertyName in $eachFieldIn)
                {
                    try
                    {
                        switch ($propertyName)
                        {
                            "positionPrimarySupervisor" {
                                try {
                                   $managerDN = $userRec.Get($propertyName)
                                   try
                                      {
                                          $supervisor = $Context.BindToObjectByDn($managerDN)
                                          $value = $supervisor.Get("cn")
                                      }
                                      catch
                                      {
                                          $value = ""
                                      }
                                }
                                catch {
                                   $value = "" 
                                }
                            }
                            "cn" {
                                # format name as lastname , firstname middleinitial if possible, otherwise as lastname , firstname
                                $lname = $userRec.Get("sn")
                                try {
                                    $fname = $userRec.Get("preferredFirstName")
                                }
                                catch {
                                    $fname = $userRec.Get("givenName")
                                }   

                                try {
                                   $mname = $userRec.Get("middleName") 
                                }
                                catch {
                                   $mname = "" 
                                }

                                $value = $lname + " , " + $fname + " " + $mname
                            }

                            Default {
                                $value = $userRec.Get($propertyName)
                            }

                        }  # end switch $propertyName

                    }
                    catch
                    {
                        $value = "" # The property is empty
                    }

                    $reportRecord | Add-Member -Name $propertyName -Value $value -MemberType NoteProperty
                } # end foreach propertyname
                $report += $reportRecord

                $count++
        }  # end foreach user
    }
    finally
    {
        $userResult.Dispose()
    }

    #$report | Export-Csv $exportFile -NoTypeInformation  *** Uncomment this line to export with the Header row
    $report | ConvertTo-Csv -NoTypeInformation | select -Skip 1 | Set-Content $exportFile

    $Context.LogMessage("Total records exported = " + $count, "Information")
}
catch  #catch any unresolved errors
{
    $ErrorMessage = $_.Exception.Message
    [string]$LineNumber = $_.InvocationInfo.ScriptLineNumber
    [string]$Offset = $_.InvocationInfo.OffsetInLine
    [string]$errLine = $_.InvocationInfo.Line

    Write-Error $ErrorMessage
    $Context.LogMessage("At Line #: " + $LineNumber + " at char " + $Offset, "Error")
    $Context.LogMessage("Executing: " + $errLine, "Error")  
}

ExpandCopy code

---

I copied and pasted the code supplied by Support above in it's entirety and it now finds and exports 0 records.

---

Hello,

We've fixed the issue with the script. Copy an updated version in the post above.

---

Thanks!!! That works... Any idea of when this will be fixed so that a workaround is no longer needed?

---

Hello,

A fix for the issue is available in Adaxes 2016 released today. Starting from this version, the IAdmBusinessUnit::Members method no longer returns unmanaged user accounts. You can download it here: http://www.adaxes.com/download_direct.htm.

Upgrade Instructions

Complete list of new features and bug fixes