XER File Slow To Import to P6? Try Cleaning POBS Data

The other day one of our Plan Academy members called our support line with an unusual P6 problem. I’m always happy to help out our members solve P6 problems, even when it’s 4:30pm on the Friday afternoon before I’m off on a full week of vacation….

OK. Full disclosure here.

I had already powered down my laptop for the weekend and was just about to grab something ice-cold and carbonated when the phone rang. I was hoping it was something we could solve quick or that I had already written a solution for. I was tired, and it was hot in the office. The patio was calling. This wasn’t how I was expecting to wrap up my week. Oh well.

The call was was from Doug (not his real name) from Alberta, a Plan Academy member who works for a large national construction company…and he was having problems with an XER file he was sending to a client. Doug’s client was trying to import Doug’s XER File, but it was taking hours to import Not an exaggeration – Doug said it took 6 hours for the client to import his P6 XER file successfully. Something was clearly wrong.

Sometimes you get those problems that just can’t stay away from because it’s something you haven’t seen before and curiousity just takes over. That’s what happened here for me. I was thinking “6 hours to import? How can that possibly be?”

So I made a bargain with myself. If I could help Doug in the next 30 minutes (by 5pm), then I would reward myself with not one, but TWO ice-cold carbonated bevvies.  I would be on vacation after all and sometimes you have to motivate yourself!

Now Doug described his project schedule to me. It was a few thousand lines – not uncommon – and it was both resource and cost-loaded. He and his client were using Primavera P6 Professional v8.4.

I did some quick research online about slow importing of XER Files, but there wasn’t any obvious leads there. Here’s what I suggested to Doug.

XER File Slow To Import Into Primavera P6? POBS XER File Data May Be the Culprit

Doug’s XER File was clearly taking too long to import. In this case, I can see 2 things that might need addressing:

1. The XER File is is having issues.
2. The client’s machine or database is having issues.

My experience is that most machines and databases don’t run into slow performance issues like they used to 10+ years ago…back in the day when you had too many background processes running on your PC and those programs would grind the processor to a halt. So I was thinking there was likely something going on with the XER file.

Here’s what I recommended to Doug:

A) Rather than use the XER File Format, try the XML format that P6 supports instead. 

The XML format functions much like the XER File format. There are some differences, but the project will come across regardless. If the XML file imports quickly, then there’s a problem with the XER file.

B) Remove the POBS Data from the XER File.

This solution was suggested recently on PlanningPlanet where this POBS issue came to my attention (and thanks to the smart P6 experts who suggested it there). I’ll outline the steps to easily removing POBS data soon, including a tip that will be a major timesaver, but first, what did Doug do?

Doug opted to try suggestion B). By removing the POBS data from his XER File, we found that he was able to reduce his file’s size from 25Mb down to 2Mb! Wow! That’s a lot of POBS data!

For Doug, removing the P6 POBS data also reduced the import time from 6 hours to >2 minutes! That’s an exponential improvement!

Yes, we got Doug fixed and I made it to the patio and kicked off a great vacation week. But I’m not finished with this POBS issue. I’m not completely sure why it’s cropping up right now, but I’ve seen enough online grumblings about it to want to tackle it in detail.

So let’s dig in and get down to the bottom of POBS data and XER files in Primavera P6.

What is POBS Data?

Since this is a fairly new issue with P6, I’ve done some homework around the POBS data issue.

Every Primavera P6 database has a POBS table. Looking at the Primavera P6 database schema docs, POBS stands for Performing Organizational Breakdown Structure. The POBS data isn’t used (yet) by Primavera P6 Professional or Primavera P6 EPPM. Here’s what an Oracle Support doc titled “What is the POBS Table Used for and What Data is Stored in POBS? (Doc ID 1959228.1)” had to say:

Functions related to table POBS have not been implemented yet so the table has not been put to use. The table may be removed in a future release.

If there is data in this table, then it’s probably present in the sample database and not inserted during operations performed in the application.

That doc says it applies to Primavera P6 EPPM v7.0.3.0 or later.

What are the problems with POBS data & XER Files?

Another support doc discussed how the POBS table is getting overly large and affecting performance of the application. Let’s cover that issue another time.

However, at least 1 Bug has been logged for performance issues related to POBS data (Bug 20800033 : EXCLUDE POBS DATA FROM XER EXPORT & IMPORT OPERATIONS), and the Bug had this to say about XER import issues:

We do not utilize the POBS table yet we export/import the data from this
table when completing XER Export/Import.  The XER export/import should be
written to exclude this data with XER export/import operations of P6
Professional.

So essentially, here’s what we know:

• POBS data exists in the POBS table in the P6 database
• POBS data is exported and imported via XER file
• POBS data isn’t used for anything
• Too much POBS data can affect performance of import (and maybe export)

I have a hunch that POBS data is related to Primavera Cost Manager, a now-defunct cost management solution that was once part of the Primavera suite years ago. Primavera Cost Manager integrated tightly with P6 Professional and looking at documentation that still exists, there was a concept of a Performing Organizational Breakdown Structure in Cost Manager. But alas, only Oracle can confirm this.

How Can I Clean POBS Data From My XER File?

[NOTE: a simple alternate solution is provided at the bottom of this post]

We’ve already shown you how can you clean global data from XER Files with the XER File Parser, but this solution is much simpler. Let me start by saying you can do this the  s..l..o…w  way or the quick way (my way). Let me show you how it’s done.

First, export your project to an XER file.

Since XER files are actually text files, you can edit them with a text editor like notepad. Here’s what one looks like when you open it up:

We’re going to delete a section of data from the file. To denote a new section, look for a “%T”. To find the POBS section, look for “%T    POBS”. That indicates the start of the POBS data in your file. You should see it somewhere near the top of the file in first 100 lines. Above, you can see it on line 29 of the file.

Deleting POBS Data

Basically, the process is to delete all lines starting from (and including) the “%T    POBS” line, until the next “%T” in the file. Do not delete the next “%T” line.

So if you’re using notepad, you can simply highlight this huge swath of text and hit delete.

In my example above, there’s less than 200 lines to highlight so it’s pretty easy.

But you’ll see that the amount of POBS data isn’t consistent. Here’s a short 30-activity project schedule as viewed in a text viewer.

The image above might be more typical. The POBS data spans from line 29 to line 4412. Most of this file is POBS data!

So if you’ve got a huge file and a huge amount of POBS to clear out of your XER file, let me know you the quick way to clean it.

The Quick Way to Clean POBS Data From a P6 XER File

Let’s begin. First, you’ll need some better text editing tools than simple Windows Notepad. I’m recommeding Notepad++; free and easy to use. It’s worth the download!

1) Make a backup of your XER File first. Edit your file in Notepad++.

2) Find the “%T    POBS” line in your file – somewhere in the first 50 lines. Note what line number this is, you’ll need it later.

3) Now, let’s find the next “%T” line. Place your cursor at the end of the “%T    POBS” line. Hit CTRL+F (or use the Search menu) and search for the next “%T” occurrence in the file.

4) Make a note of the line number you’ve been taken to. Mine was line 4415.

5) Now we’re going to use some tools to highlight the POBS data since we know the line numbers. Go back to the “%T    POBS” line. Place your cursor at the beginning of the line. Now Goto Edit -> Begin/End Select. This essentially tells Notepad++ to start highlighting here.

6) Now hit CTRL+G (or goto Search -> Goto). Make sure to choose Line and type in the line number from step 4) – for me 4415.

7) Now that you’re on the next %T line, tell Notepad++ to end the highlighting here. Goto the Edit menu and uncheck Begin/End Highlighting.

8) Now all of your POBS data is highlighted. Simply tap Delete or Backspace to remove it from the file. Make sure you delete the %T POBS line as well, but not any other lines.

9) Save the file and import.

For the xer file I used in this writeup, removing the POBS data took it from 4489 lines down to 102 lines. And it reduced the file from 564KB down to a mere 21KB.

Why Are Primavera P6 Users Having Issues With POBS Now?

That’s a tough question.

Reports from Oracle say that once Primavera P6 POBS data is in your database, it will be exported along with an XER File. The receiving database will then “be infected” with this data. The data seems to build up in the database (something I’ll cover fixing in my next post) and continues to grow as files go back and forth.

But why is this suddenly an issue now?

One theory I have is that a recent release, possibly Primavera P6 v8.4, had POBS data in the sample database that is often setup at install time. I’m choosing v8.4 as it was referenced in the Bug I mentioned earlier in this post. Users often use the sample database as a production database and the data could be propagating from there.

Whatever my theories are, these POBS issues seem to be appearing more and more.

Conclusion

If you’re having difficulties with performance importing or exporting Primavera P6 XER files, try cleaning the POBS data from them. Yes, you can use windows Notepad, but if the file is large, you’ll be scrolling for an hour. Try using the techniques I’ve shown above along with the Notepad++ text editor to save you time.

I fully recognize that what I’m offering here isn’t truly a permanent solution. Constantly editing XER files to have a reasonable import time isn’t viable and it is a product issue that needs to be addressed by Oracle. Here’s hoping this gets you by in the meantime.

In my next post, I discuss the POBS table in your Primavera P6 database and strategies to keep it clean or empty. Read it now – How to Clean Your P6 POBS Table On MSSQL & Oracle DB.

Update – 2016

It turns out that using notepad is really too tedious, although some commenters on this post had some great suggestions for making that process easier. But let me make life even easier by recommending a simple utility created by Plan Academy reader and senior planning engineer Ahmet Kochan. Ahmet’s POBS Cleaner is dead simple to use and free! POBS Cleaner can clean 1 or more XER files at the same time. It creates a newly cleaned version of your XER file with the extension __cleaned.xer in case you ever need the original file.

There are some other tools on the market that clean POBS from XER files, but Ahmet’s tool is free and so simple to use that I highly recommend it.

Download POBS Cleaner

Enjoy! POBS Cleaner is even packaged here with a test project so you can see how it works before you use it on a real project.