In 2016, the Jama Software team proudly announced that we had received a certification from internationally-recognized testing body TÜV SÜD. Jama Connect™ was certified as a software tool for development of safety-related products according to ISO 26262 (up to ASIL D) and IEC 61508 (up to SIL 3). It was especially noteworthy, as Jama Software was one of the first vendors to be both SaaS and agile to have received this certification.
Three years later, we are excited to announce we have extended the scope of our certification from TÜV SÜD. Jama Connect is now also certified as a software tool for development of medical devices according to IEC 62304 and railway applications according to EN 50128.
This new certification gives medical device developers and railway application developers confidence that Jama Connect has been evaluated and qualified for defining, building and testing products that have to meet critical functional safety requirements.
We recently talked with Christian Nowak, Functional Safety Expert at TÜV SÜD, to discuss what is required to receive such certifications and what they mean for our customers.
Jama Software: Can you explain, generally, how the certification process is completed?
Christian Nowak: For a software tool certification, we are focusing our assessment on the development processes and the validation approach and evidences provided by our customers. An important activity is the on-site audit at the customer’s premises. The first audit was conducted in 2016 as part of the original certification and we performed a three-day re-audit in June 2018 at Jama Software’s headquarters in Portland, OR.
During the initial audit we looked at the organization’s processes in the light of the functional safety standards’ requirements for developing and maintaining safety-relevant software. The recent re-audit included sample checks to see if these processes are being followed based on the evidences created.
We also discussed and assessed modifications and improvements of processes, which play an important role for verification and validation especially in the context of the Agile development approach at Jama Software.
JS: What is required in addition to the on-site audit?
CN:The on-site activities are usually supported by off-site reviews of the documentation evidences generated by the customer. Due to the agile development approach at Jama Software leading to frequent releases, our assessment approach had to be adapted and is following this pace.
In other words, we are regularly assessing the modifications Jama Software is applying remotely and updating our certification report accordingly. In a way, our assessment approach has, in this case, also become “agile.”
JS:What does this certification mean for our customers? How can they benefit from these certifications?
CN: Every Jama customer who is attempting to adhere to the mentioned standards for developing safety-related systems, hardware or software must provide documentation evidences addressing the requirements defined in the standards.
Those requirements also consider the software tools that are used for the development of safety-related systems, hardware or software. The idea is that systematic faults should be avoided not only in the actual developed systems, hardware or software, but also in the software tools used for the development.
By undergoing a successful third-party certification by an accredited testing body like TÜV SÜD, Jama Software demonstrates that they are following adequate development processes and performing adequate validation activities for preventing systematic faults.
Thus, Jama customers can use the TÜV SÜD certificate as an argument for software tool qualification in projects where increased confidence in the software tool is required. They do not have to spend all the efforts for qualifying the tool themselves; they only have to make sure that they are following the safety manual that Jama Software is providing for each release.
JS: What is the value for organizations developing products in accordance with these standards?
CN:In some industry fields, functional safety standards are mandatory to be complied with – in others, product liability is a main driver. In general, the quality, reliability and of course the safety of those products are being improved, which helps avoiding recalls, sanctions, and worse – consumer injury.
JS: How long is the certificate valid for?
CS: Generally, a TÜV SÜD functional safety product certificate is valid for five years. During this timeframe, TÜV SÜD is however regularly monitoring the adherence to the requirements by accompanying the agile development remotely as mentioned before and by returning every two years for an on-site audit.
JS:Is TÜV SÜD involved in the development of the functional safety standards?
CN: Yes, TÜV SÜD is actively participating in the standardization committees. Please note that just recently the second edition of the automotive functional safety standard was released (ISO 26262:2018).
JS: How long does the tool certification process take, on average?
CN: Well, this depends on the maturity of the existing development processes, the complexity of the tool and the experience of the company with functional safety when we start with the assessment. I would say the initial certification can be achieved within six months, but it can also take much longer if many iteration loops are required.
To learn more about how Jama Connect can help your team simplify compliance, streamline development, and speed time to market, download our solution overview.
In today’s competitive market, automobile makers are racing to define the future of transportation. And given the complexity of modern, connected automobiles, it’s imperative that vehicle safety is adequately accounted for in the product development process.
Plus, as vehicles become more complex — e.g. autonomous driving and connected systems — so too are the standards for emissions, fuel economy, functional safety, cybersecurity, and system designs.
That’s why we’ve partnered with LHP Engineering Solutions (LHP) to ensure our visionary clients comply with all relevant functional safety and cybersecurity standards — like ISO 26262 and SAE J3061 — by seamlessly integrating compliance into the product development process.
Founded in 2001 with the mission to make safer products, LHP provides a variety of engineering services and products to assist customers in speeding up their product development cycles and solving product design and testing issues.
We recently spoke with the LHP team to talk about modern challenges in complex product development, and how this new partnership between LHP and Jama Software can help. Let’s dive into what customers can gain from this partnership:
Jama Software: Can you give us a rundown of the state of the automotive industry? How is it different than 10 years ago?
LHP: The next generation of automobiles are increasingly incorporating modern electronic technologies, from on-board diagnostics to engine controllers to infotainment systems to driver assist systems. As technology advances, the trend is to partially/fully automate vehicles. While some new features are entertainment- and convenience-based, the trend for autonomous vehicles is, to a large extent, functional safety related. The end goal is to reduce the number of deaths on public roadways by providing vehicles with the ability to recognize and avoid hazards or security threats.
Safety and security are the two biggest barriers to innovation and we’re helping companies overcome those barriers. The biggest reason why we don’t see self-driving cars everywhere is because, before that happens, we must prove that they’re not going to harm people, that they’re safe, and that the software can’t be compromised and used for unintended usage.
JS: What are some of the biggest challenges facing product development teams in the automotive industry today?
LHP: Compliance to ISO 26262 and SAE J3061 involves a change in culture that is difficult for established product development teams to implement. Part of this change means looking at the product(s) being developed differently, and part is a change in infrastructure to better control the product development lifecycle and the development artifacts.
JS: How does leveraging the partnership between LHP and Jama Software help customers when it comes to overall functional safety as well as complying with ISO 26262?
LHP: ISO 26262 complements good systems engineering practices by requiring that hardware and software safety concerns be addressed and documented in a systematic way throughout the product development lifecycle. In the past, safety design was considered part of general requirements activity. But merely identifying and tracing requirements in the software and hardware designs is not enough. The common practice of hardware and software teams working in silos will not guarantee the level of safety coverage required by ISO 26262.
Part of LHPs offering is development of data, infrastructure consultation, and process optimization. Now, thanks to our partnership with the Jama team, we can implement proper functional safety workflows in Jama Connect, with templates to facilitate the creation of data. Additionally, Jama Software customers in the automotive industry who have questions and concerns about how to use Jama Connect to support a safety lifecycle can tap into LHPs extensive knowledge and experience in functional safety and ISO 26262.
In order to demonstrate compliance with ISO 26262, you must have the ability to manage safety requirements, including traceability. Typically, our respective customers would need to address functional safety and requirements management separately. Working together, LHP and Jama Software can address both sets of concerns in concert.
Learn how Jama Software worked with TÜV SÜD on our ISO 26262 certification process, and how you can lower the costs and risks of complying with functional safety standards, by watching our webinar.
JS: What does LHP bring to the table that other requirements management platforms might not have access to?
LHP: What makes us better and unique compared to a lot of the other organizations is our wealth of knowledge. Our functional safety team actually came out of the aerospace industry with multiple decades of experience implementing safety-critical systems. The experience that they bring with them has time and time again proven to make us stand above the rest.
Just like Jama, LHP excels at custom integrations and tailoring flexible solutions for our customers. At LHP, we consult on and implement the latest automotive industry practices to ensure vehicle systems are safe, reliable, and secure. Customers come to LHP for our deep knowledge in embedded controls, integration support, and overall implementation of functional safety and cybersecurity processes.
Proving compliance with functional safety and cybersecurity standards like ISO 26262 and SAE J3061 requires a harmonious balance of processes, people, and tools. And together with LHP Engineering Solutions, Jama Software is helping automotive companies safely and confidently bring the future of transportation to market.
To learn more about how to maintain compliance with automotive functional safety standards and how to avoid common ISO 26262 mistakes, download our whitepaper, “Top 15 ISO 26262 Snafus.”
https://www-dev.jamasoftware.com/media/2019/06/LHP_Functional_Safety.png5121024McKenzie Jonsson/media/jama-logo-primary.svgMcKenzie Jonsson2020-09-29 03:00:422023-01-12 16:50:17Jama Software Partners with LHP to Help Automotive Companies Comply with Functional Safety and Cybersecurity Standards
In our experience working with more than 200 medical device developers, we’ve realized how important it is to create best practices for risk management under ISO 14971, the FDA’s mandatory standard for risk assessment throughout the product development lifecycle.
In this post, we’ll outline the main clauses of ISO 14971 and explain how Jama Connect can help medical device developers build better, safer products that satisfy ISO 14971.
What is ISO 14971?
ISO 14971 is an international standard that sees risk management as a product lifecycle process encompassing the development, production, and post-production stages. Jama Connect offers a straightforward approach to managing risk according to ISO 14971 in one platform. The standard was updated in 2019, providing more guidance on risk management and adding more detailed requirements.
Managing Risks & Requirements for ISO 14971
Risk management is an inextricable part of the medical device development process. For medical device developers, risks are a core principle of product development and should be tied together in one powerful platform.
Many medical device companies continue to depend on Excel to capture risk data, but Excel simply can’t provide the end-to-end traceability necessary for satisfying ISO 14971. That’s where Jama Connect comes in: It allows teams to easily connect risks, requirements, and testing in one system where requirements and test results stay live in real-time.
Jama Connect and ISO 14971
Jama Connect guides compliance with Clauses 4 through 7 of ISO 14971, which covers how risk should be managed throughout the product development process.
Clause 4 of ISO 14971 concerns how risk is organized and administered for your product line. It requires the formation of a Risk Management Plan throughout the development lifecycle.
The Risk Management Plan is the record of a planned process for risk management: who does what and when, how risks are scored, etc. It’s a component of the Risk Management File, which contains all the outputs for risk.
Clause 5: Risk Analysis
Clause 5 of ISO 14971 requires that medical device developers identify potential hazards and hazardous situations. Each hazardous situation and its potential consequences must be evaluated. Jama Connect helps teams satisfy Clause 5 by defining device-specific hazards and capturing risk probability and severity.
Jama Connect offers risk management item templates to capture important information about the risk analysis process, including a description of the device, intended use, and the scope of the analysis.
Teams can identify and evaluate potential hazards, sequences of events, hazardous situations, and harms in a single item type.
Clause 6: Risk Evaluation
Clause 6 requires the evaluation of risk for each hazardous situation and the definition of acceptability criteria for determining when risk reduction is required. To satisfy Clause 6, teams take the inputs from Clause 5 and determine the risk level for each hazardous situation.
In Jama Connect, risk acceptability criteria can be customized for a particular product line or medical device classification in the risk management item.
Clause 7 requires risk control measures to be developed, implemented, and verified across the product development lifecycle. Risk control measures could include product design, preventative measures in the product, and labeling. Residual risk must be evaluated against acceptability criteria, and risk control measures must be reviewed in case additional risks have been introduced inadvertently.
The risk evaluation item lets users identify risk control options for a specific hazardous situation, such as inherent safety by design, protective measures in the medical device or manufacturing process, and safety information.
Risk control measures, implementation verification, and verification of risk control effectiveness can also be accounted for in the risk evaluation item. Links to system requirements and verifications in Jama Connect can easily be created from the risk item to demonstrate traceability from hazardous situations to risk controls.
Clause 8: Residual Risk
Clause 8 requires an evaluation of the medical device’s overall residual risk. If the overall residual risk is unacceptable, it must be demonstrated that the medical benefit outweighs the residual risk.
When defining risk control measures, teams can capture those measures in Jama Connect and link them directly to risks before updating the rankings to determine the residual risk level.
With traceability through all phases of risk, users can quickly identify potential pitfalls in the product development process and address them before they become bigger barriers to success.
The Bottom Line
ISO 14971 requires a cohesive, well-documented narrative of your product’s lifecycle to assure the FDA that the device is safe, effective, and compliant. Any decisions or actions that aren’t documented could keep your product from reaching the market or result in a recall.
Finding and fixing errors early in the product lifecycle saves money, speeds time to market, and improves product quality. Jama Connect allows medical device developers to review risks and risk controls holistically so that teams can operate with confidence.
From a compliance perspective, the Jama Connect for Medical Device Development illuminates the risk management and product development process, while simultaneously generating the required documentation to support that narrative.
https://www-dev.jamasoftware.com/media/2019/01/2020-06-08_ISO-14971.jpg5121024Jama Software/media/jama-logo-primary.svgJama Software2020-06-08 14:00:402023-01-12 16:50:35ISO 14971: Managing Risk for Medical Device Developers in Jama Connect®
As we enter a new decade of technological advancements, Jama Software asked select thought leaders from various industries for the trends and events they foresee unfolding over the next 10 years.
Today we’re featuring predictions from Josh Turpen, Chief Product Officer at Jama Software, who oversees the ongoing innovation and refinement of our core product offerings.
Jama Software: What product development trends are you expecting to take shape in the 2020s?
Josh Turpen: “I think there will be an increase in hardware/software solutions. There have been amazing advances in software and hardware; now it’s time to put the two together. This will finally deliver on the ‘Jetsons’ future that my father has been asking forever since I told him I wanted to be an engineer!”
JS:In terms of product and systems development, what do you think will remain the same over the next decade? What will change?
JT: “The need for clarity, consistency, and visibility in requirements and test will continue. We still need to know ‘what’ to build, and we need to explain to others ‘how’ we’ve done it. In fact, this will continue forever. What’ll change is how we do the development.
“More artificial intelligence and machine learning will be introduced into quality organizations to check both the input (requirements) and the output (tests). This will expand into development, but the innovation is happening in quality right now and I think that’s where the most ROI exists.”
JS:How do you foresee regulations shifting in certain industries over the next decade?
“I think regulation — an area many of our customers, particularly those creating medical devices, autonomous vehicles, and rocket ships, are familiar with — is coming for all but the most trivial software and hardware applications. At the point where data and safety meet, you’ll see the most regulation.”
JS:Any major disruptions to specific industries you’re anticipating in the 2020s?
JT: “Data warehousing companies — such as some of the biggest social networks and search engines — will come under a new regulatory body of some kind. The amount and quality of data is going to be of paramount importance and consumers will demand protection. We’re seeing the start of this, and I think governments will respond much the way they did with the credit bureaus in the 70s.”
JS:What sorts of process adjustments do you think development teams will need to make to be successful in the next 10 years?
JT: “Integrated software/hardware development — something which many of our customers are already heavily involved in — will require a new focus on cross-development type processes. ‘Pure Agile’ or ‘pure waterfall’ will cease to exist and a blended model that recognizes the strengths and weaknesses of both will emerge as the dominant paradigm.”
JS:What do you think will be some of the differentiators between a company surviving to see 2030, and those that do not?
JT: “Companies that are structured to accept, respond, and act on change will continue to dominate. Those that require a static customer base, employee skillset, or product deployment environment are doomed.”
JS:Where do you see Jama Software fitting in as the product development landscape evolves over the next decade, and what can our customers expect in return?
JT: “We will focus on requirements, test management, risk, and visibility across those three areas. Our solution is created by product developers for product developers. We understand the demands that our customers are working under and are constantly iterating Jama Connect™ to meet and stay ahead of where the industry is going. Customers can expect Jama Software to be fanatical about delivering value to them and increasing their capability to focus on product development, not the tooling.”
JS: Any other thoughts you’d like to share on the future?
JT: “I’m excited about where technology is going! We’re seeing advances in problem areas that have plagued humanity since the beginning of time. Communication, medical, and efficient use of resources are all areas where the promise of technology is coming true. Once we get those problems solved, we can focus on the really hard stuff, like getting the clock in my oven and microwave to stay in sync when the time changes.”
About Josh Turpen:
With a deep background in software development and consulting, Josh oversees the ongoing innovation and refinement of Jama Software’s core product offerings. Beginning as an engineer, Josh’s career has taken him from Indiana to Germany, Colorado, and Portland. His work with the U.S. Department of Defense solidified his knowledge of safety-critical systems, and the vital role requirements and risk management plays within them. Having led product and engineering organizations, with teams distributed across the globe, Josh understands the daily challenges our customers face in a constantly changing marketplace and the tools they need to be successful.
Significant growing and shifting regulations are already underway within the medical device industry. Learnhow teams can stay ahead of the competition in our eBook, “Conquering Connectivity, Competition & Compliance.”
https://www-dev.jamasoftware.com/media/2019/12/2020Blog-Series-02-2.png5121024Jama Software/media/jama-logo-primary.svgJama Software2019-12-12 12:40:592023-01-12 16:51:142020s Predictions: Get Ready for More Regulations
Today’s move-fast-get-it-to-market-yesterday product development culture puts a lot of pressure on companies to innovate quickly. Such circumstances can make defined processes and comprehensive documentation look unsexy and uncompetitive… even when they’re in the best interest of the organization.
In October, Jama Software and kVA by UL co-hosted a kVA Automotive Lunch & Learn at the Hyatt House in Silicon Valley. kVA by UL is a technical and management consulting group focused on functional safety and the ISO 26262 standard. Bill Taylor, Managing Director of kVA, spoke to a group made up primarily of automotive industry engineers, many of whom are working on autonomous vehicles.
Taylor presented on the topic of “Creating a Safety-First Culture in Automotive Development,” but the points he made could easily be applied to any complex product development where public and/or user safety are a primary concern. Here are five key take-aways from Taylor’s presentation.
Don’t Fall for the Smart Folks Fallacy
When we get on an airplane, do we trust that our pilot and co-pilot are experienced, well-trained professionals? Do we assume they really know what they’re doing and that they’ve done it many times before?
Of course, we do. So, why do the airlines — and the military, and every other aviation employer —make their pilots use checklists?
Learn how to mitigate common ISO 26262 mistakes with our whitepaper, “Top 15 ISO 26262 Snafus.”
It’s so they don’t have to think about it. The checklist is there so a distraction doesn’t cause the pilot to miss or forget a small but crucial step in their procedure.
But many who innovate for a living — especially those who face pressure to innovate rapidly — don’t like checklists. Checklists feel cumbersome, tedious, slow, and perhaps even antiquated. They’re constraining. They don’t let you work the way you want. Checklists force you to work to their dictates.
But checklists are great for safety, says Taylor. They force you to take all the prescribed steps.
Taylor warns against a phenomenon he sees often in the automotive and tech industries, which can roughly be described as the “Smart Folks Fallacy.” He describes it with a fictional conversation, which we’ll paraphrase:
“Hey, who’s keeping us safe?”
“Oh, don’t worry, we’ve got some smart folks over there.”
“Yeah, but how do we know we’re safe? Have we written good requirements? Have we analyzed them? Have we done direct testing? Do we have traceability throughout our process?”
“Nah, we haven’t done any of that. But those folks over there are really smart. So, we’ll be fine.”
Unfortunately, even smart people make mistakes. They’re under pressure to deliver. They get wrapped up in their immediate priorities. Even a smart person might miss something that’s tiny but critical.
Safety Relies on Process and Culture
Taylor says it’s hard to standardize culture. A true safety culture requires more than just following a set of rules, regulations, and standards. But a framework can be articulated.
In that framework, Taylor would expect to see a set of processes designed to comply with applicable safety standards. He would want to see process documentation that, among other things, describes:
The steps of the safety process
Any templates that are to be used
The tool set that will be used
Outputs that get reviewed, and who reviews them
The qualifications necessary to be a reviewer
Failure to follow a sound safety process leads to disaster, says Taylor. He maintains that in every accident report that makes the news, there is someone who says, in effect:
“I realized this could be a problem. I told my boss, and he kind of understood, but not exactly. Anyway, we were crazy busy, etc., etc.… It just got pushed aside.”
You need a system and a culture in place that will turn that potential problem into a documented safety anomaly — that will put it on the radar and make sure it gets addressed, says Taylor.
A Safety-First Culture Needs Police Officers
Just like governments need a police force to enforce laws, companies must invest in safety personnel with the authority to prevent potential hazards from being released into the field.
This doesn’t mean putting safety managers at the top of the pyramid, Taylor says. Instead, he likens it to Toyota’s famous quality system, where every plant worker has the authority to stop the production line if they spot a flaw. It’s a huge expense when it occurs, but it can save much greater expense down the line by avoiding costly recalls and lawsuits. It also makes everyone realize quality assurance is a part of their job.
It’s the same with safety. When a problem is a safety issue, the safety team must have the power to say, “Stop! We have a process to communicate and address this issue. We have the authority to make things stop until the issue is fully resolved.”
Learn how Jama Software worked with TÜV SÜD on our ISO 26262 certification process, and how you can lower the costs and risks of complying with functional safety standards, by watching our webinar.
Embrace the Idea of “Nothing Happening”
Taylor says safety engineers have a weird point of view: They’re inspired by the idea of nothing happening. That’s because, when people buy your product, they expect that nothing bad will happen to them… ever. If something bad does happen to them, and it’s your fault, they’re likely to sue you.
“Someone has to take ownership of this idea,” Taylor said. “There must be a group that says to themselves, ‘While the rest of the organization is innovating and creating amazing things, I’m going to make sure that nothing bad happens, and I can be motivated by that idea.’ It means you’re not out there making the thing be better and more awesome, you’re just trying to make it work in a way that doesn’t kill anyone.”
“And usually,” Taylor continued, “Your CEO says, ‘I thought it was just assumed that it wouldn’t kill anyone.’ And you say, ‘Yes, but someone has to be in charge of that. Someone needs to own that.’”
The Manifesto for Agile Software Development values “individuals and interactions over processes and tools” and “working software over comprehensive documentation,” he said. Safety, on the other hand, cannot be assured without a commitment to process and documentation. “Software observed to be ‘working’ can often be fundamentally unsafe,” Taylor noted.
Taylor went on to say that where Agile tries to reduce friction, safety values friction. Friction is how we maintain control. Without friction, everything slips out of our hands.
“If our process has no friction, it’s out of control,” he said. “Agile is not wrong. Agile is wonderful for making progress” but it’s not sufficient for ensuring safety.
In complex product development, Taylor says, we don’t see a lack of smart people. What’s often lacking is a commitment to process and the tools that enforce that process.
We need to make allowances for Agile and a safety-first culture to coexist. Use Agile to make rapid progress. But allow safety to apply its processes — and apply the brakes, when necessary.
In other words, we need to accept the checks and balances of the safety process. Taylor summed it up this way: “Value friction. Value slowness. Take ownership of ensuring safety.”
To further assist in mitigating risks in the development process, maintaining compliance with automotive functional safety standards, creating a safety-first culture, and staying updated on the changes to ISO 26262, check out our white paper, “The Impact of ISO 26262 on Automotive Development.”
https://www-dev.jamasoftware.com/media/2019/11/Safety-First-Culture.png5121024Jama Software/media/jama-logo-primary.svgJama Software2019-11-26 04:30:312023-01-12 16:51:165 Ways to Build a Safety-First Culture
Reviews play a key role in successful product and systems development, helping to ensure a new project meets stakeholder, market, and compliance requirements.
By taking an iterative and collaborative approach to reviewing requirements and tests in real-time, Jama Connect™ Review Center improves stakeholder alignment, reduces lengthy review cycles, and eases the path to compliance.
To that end, we are excited to announce today that we have made significant improvements to Review Center, an already beloved feature of the Jama Connect platform, which are available now to our customers.
Within Jama Connect Review Center, users can now streamline reviews and ease the path to regulatory compliance thanks to:
A new Review Center Wizard that streamlines review set up for particular objectives
Simplified application, management, and accessibility of electronic signatures for reviews
Easier configuration of review settings, such as ones used to apply electronic signatures per FDA 21 CFR Part 11 to ensure they can be properly referenced and signed off on across different teams.
Additionally, customers can now add electronic signature roles, prevent reviews with e-signatures from being deleted, view review activity history for audits, and better set expectations for participants based on their role.
Streamline the review process with clearly identified participant roles and electronic signature settings.
These enhancements make the review process more straightforward and help customers meet compliance requirements for standards like FDA 21 CFR Part 11.
In the coming months, we’ll continue to invest in review and approval enhancements to improve collaboration and further address 21 CFR Part 11 compliance in Jama Connect Review Center.
Review Center Departs from Traditional, Document-Based Reviews
Traditional review processes often stifle collaboration and focus on a document-based approach, resulting in misalignment, long review cycles, versioning issues, and an abundance of unnecessary meetings.
By creating a centralized place to manage and collaborate on reviews, Jama Connect Review Center eliminates the need for lengthy, in-person meetings or back-and-forth emailing, making reviews more efficient and scalable.
Gain visibility into all major review activities.
Michelle Seitz, Senior Business Analyst at MediSync, believes one of the most significant benefits of Jama Connect is the reduced time and effort it takes to complete review cycles using Review Center.
“My favorite part of Jama right now is when I get a collection of requirements to send out for a review,” Seitz says. “It is so much better with Jama. It’s the first time I’ve had a product that works as seamlessly as Jama does to produce a review and get feedback without having to do all the track changes and stuff that we used to have to do in Microsoft Word.”
And the results of the improved review process speak for themselves.
MediSync says that Jama Connect has saved the organization 80% of planning time that previously would have been wasted on meetings, sorting through versioned documents and emails, and consolidating feedback in review cycles.
Global healthcare leader Grifols says that Review Center has helped it shorten review cycles from three months to fewer than 30 days, while reducing budget overruns. They estimate savings of over 80 hours per project in medical device development thanks to Jama Connect Review Center.
As an organization, Jama Software is committed to improving stakeholder alignment, reducing lengthy review cycles, and easing the path to compliance by providing our customers with a modern approach to reviews and collaboration. And with Review Center’s new improvements and feature enhancements, we can confidently say that we are doing just that.
https://www-dev.jamasoftware.com/media/2019/10/Jama_Connect_Review_Center.png5121024Jama Software/media/jama-logo-primary.svgJama Software2019-10-29 06:00:352023-01-19 13:01:19Introducing the New and Improved Jama Connect Review Center
Self-driving vehicles are coming. There’s a certain sense of inevitability. Mentions appear almost daily in the news with players such as Tesla, Uber, Google/Waymo, and Apple spending millions on development. Yet the public is uncertain of the value and safety of such vehicles.
If autonomous vehicles (AVs) are to find acceptance, the industry must produce vehicles worthy of trust. The characteristics on which trustworthiness depends, and the path for trustworthy AV development, are described below.
Figure 1- Mckinsey & Company Self-Driving Vehicle Revolution Exhibit
The Value of AVs
Just because such vehicles may be possible, is this evolution a good (or, the right) thing to do?
If implemented correctly and carefully, the move to fully-autonomous vehicles can provide real gains for society.
Highway safety – Automakers and civil engineers have made great strides in past decades in reducing highway injuries and deaths. Today’s cars include crumple zones, airbags, collapsible steering columns, and anti-lock brakes. Roadways have improved-traction surfaces, energy-absorbing barriers, and better signage and alert systems. The driver, however, remains the largest contributor to highway fatalities in the U.S. with 30% due to excessive speed, 30% from driving under the influence, and 16% attributed to distracted driving.
In the AV world, vehicles do not suffer from a human driver’s inattention, bad attitude, or inebriated operation. Instead, vehicles are under constant electronic guidance, in continual communication with the supporting infrastructure (e.g. GPS), and in a perpetual state of monitoring surrounding vehicles, obstacles, and environmental conditions. Vehicles, as a group, maintain proper positions and adequate spacing, resulting in significantly fewer injuries and deaths.
The functional safety standard ISO 26262 is a critical component of automotive development. Jama Software and LHP have teamed up to give developers an overview of the standard, and highlight its recent changes.
Traffic flow and roadway capacity – Highways and city streets can be expanded only so much to accommodate growing populations. AVs can make better use of available roadways.
In slow-moving traffic, human drivers tend to be selfish and jam too tightly together (“If I leave three car lengths open, everyone will pull in front of me”); yet, that space is exactly what is needed to allow more freedom to enter a freeway and to change lanes. AVs take the emotion out of driving decisions. On open, flowing highways, the safe following distance for human-operated vehicles could be reduced by a factor of five or more for AVs in close communication, thereby allowing more vehicles per mile.
Energy consumption – With communication between vehicles, the need to brake by one AV could be signaled to those nearby, allowing the group to slow as a whole and avoid the accordion effect which afflicts human-driven cars. This sort of coordinated action enables smoother transitions in speed and better energy usage.
Transport availability – Though services such as Uber and Lyft can provide door-to-door transportation for those unwilling or unable to drive, they do not always fit the situation. AVs can carry young teenagers to their destination without parents worrying about the integrity of a service driver. For people with physical limitations (blindness; health problems; physical disability), the AV can provide transport that is both familiar and appropriately outfitted to suit any special needs.
Simple convenience – The AV eliminates the need to drive. Passengers work or socialize as the vehicle moves along. Shoppers step out at the front of the store while the vehicle searches out a parking space on its own.
Public Response
Though today’s consumers recognize the potential advantages of AVs, they are still cautious. Recent surveys (in 2017 and 2019) by the American Automobile Association showed that 55% of U.S. drivers feel that most cars will have the ability to self-drive by 2029. Yet, today, over 70% fear riding in a self-driving car and 54% feel that their safety is at risk if sharing the road with AVs. In a 2017 survey, insurer AIG found that over 70% of U.S. respondents had concerns about AV security (hackers taking control of vehicles) and privacy (loss of personal data).
As with previous technological evolutions, AVs cannot be pushed on the public; instead, people must find enough comfort to accept or even demand new devices, especially when their safety is involved.
Elisha Otis installed the first passenger elevator in 1857. It was more than a decade before potential passengers exhibited significant trust even though early elevators were manually controlled by a human operator who opened and closed the doors, put the car in motion, and brought it level with the floor where people were to exit. The driverless elevator was created in 1900, yet it was the 1940’s before it started to see wide acceptance.
Trust in elevators was built slowly with the addition of various devices intended to ensure safety (springs and latches that would catch a falling elevator; interlocks on doors preventing opening onto empty shafts) and comfort (a soothing voice issuing from speakers to calm the nervous rider).
Could collaborating with competitors boost autonomous vehicle development? Read our blog post.
Building Trust
Self-driving cars will likewise require demonstrations of safe operation, time, and familiarity to find trust and acceptance.
The process has already begun with the current rollout of driver assistance features such as lane departure warnings, adaptive headlights, and collision avoidance systems. Continued incremental steps will further enhance driver/passenger confidence in the technology’s abilities.
Another stage may be demonstration of AV performance in closed environments such as providing public transportation at airports or on a university or commercial campus.
A good user interface may also help. Studies at Intel, Stanford, and Northwestern University all suggest that trust is improved by visual or audio feedback. Passengers find more faith in the AV’s competence if the vehicle advises why it is taking specific actions (such as voice announcing that the vehicle is slowing for a pedestrian).
Unfortunately, trust is hard-won and easily lost. Two high-profile fatal accidents in 2018 involving self-driving technology raised immediate concerns in the minds of the public and governments.
Vehicles Worthy of Trust
To avoid such incidents and maintain growth in public acceptance, the makers of AVs must build systems that are worthy of trust.
This autonomous evolution is much more complex than previous technological advancements. AVs must be able to detect and respond to numerous factors including obstacles, traffic signals, and weather conditions. Humans can distinguish between a tumbleweed and a child entering the road. Humans can contend with other vehicles which might or might not be self-driving. However, autonomous systems are much better at optimizing the driving experience to vastly increase efficiency and safety. For example, the safest distance for following a vehicle is where the second one is nearly touching the bumper of the one in front of it. This level of driving accuracy cannot be achieved reliably with humans but may well be within the realm of possibility for autonomous systems. However, it is an enormous undertaking to place such responsibility and discretion into an electronic system with expectations of safe, lightning-fast, dependable decisions.
This AV trustworthiness requires holistic consideration of five characteristics:
Safety – Ensures that a system operates without unacceptable risk of physical injury or damage to the health of people.
Security – Protects a system from unintended or unauthorized access, modification, or misuse.
Reliability – The ability of a system or component to perform its required functions under stated conditions for a specified time duration.
Resilience – The ability of a system or component to maintain an acceptable level of service in the face of disruption. The main purpose of resilience is to prevent or at least reduce any serious impact of a disruption to the system by damage or loss of operation.
Privacy – Protects the right of individuals to control or influence what information related to them may be collected and stored and by whom and to whom that information may be disclosed.
Figure 2 – Industrial Internet Consortium Security Model
These elements are generally considered as separate specialties, but should be engineered and managed as one integrated discipline because, if one piece is compromised, the overall integrity and trustworthiness of the system are undermined.
Convergence, Standardization, and Legislation
Work is progressing on each of the five characteristics to varying degrees but, unfortunately, in independent silos and in somewhat disparate directions. Though initial divergence is common with new technologies, the industry must begin to converge and standardize.
The airline industry and railroad systems both have strict standards and regulatory bodies. Automated highway vehicles must reach the same level. Currently, the industry has reached no agreement on conditions, abilities, or baselines that must be in place before an autonomous/connected vehicle is placed on the road.
A Fortune 100 semiconductor company is navigating the growing complexity of autonomous vehicles with modern requirements management.
A start has been made. ISO 26262 (Road Vehicles – Functional Safety) defines a process that will lead to high quality (trustworthy) results IF and only IF the industry can define the boundaries and requirements to be achieved. In autonomous driving, the variables and scenarios may number in the billions and are potentially non-static if artificial intelligence is used in design.
In addition, two new standards are under development:
ISO 21434 (Automotive Cybersecurity) which builds on, and works in concert with, SAE J3061 (Cybersecurity Guidebook for Cyber-Physical Vehicle Systems)
ISO/PAS 21448 (Road Vehicles – Safety of the Intended Functionality, or SOTIF) that attempts to provide guidance on design, verification, and validation measures to avoid risks resulting from functional insufficiencies and foreseeable misuse.
If the industry cannot move itself to effective standardization, the combined action of litigation, liability, and/or government regulation will likely intervene. This has happened before. In Ralph Nader’s “Unsafe at Any Speed”, his 1965 commentary on the automotive industry’s lackadaisical approach to safety caused a public uproar which led to the passage of seatbelt laws across the U.S. For AVs, a lack of convergence and standardization could likewise lead to design by legislation.
Figure 3 – The convergence of safety and security standards
Ecosystem for Trustworthy AV Development
LHP Engineering Solutions provides expertise to the automotive industry on topics including embedded controls, telematics, and data analytics. LHP has defined an ecosystem consisting of seven necessary focus areas that, if pursued together, will place the development of autonomous vehicle technologies on the right track regarding safety, standardization, and automation.
AUTOSAR (AUTomotive Open System ARchitecture) – Founded in 2003, AUTOSAR is a “worldwide development partnership of vehicle manufacturers, suppliers, service providers and companies from the automotive electronics, semiconductor and software industry.” The association aims to standardize the software architecture for automotive electronic control units. This creates the opportunity to automate software testing which should result in improved software quality and reliability.
Functional Safety – Safety in autonomous driving is of the utmost importance and is key to trustworthiness. Functional Safety relates to a system or its components operating correctly in response to inputs, including the detection, mitigation, and/or correction of malfunctions.
Cybersecurity – Trustworthiness cannot be realized without a strong foundation in cyber security. Though systems may be designed for safety, resilience, and reliability, the public may experience havoc and hazards if those systems are compromised by a malicious series of attacks. Cyber security provides the basis for assuring the integrity of the safety, reliability, resilience, and privacy characteristics of automotive systems.
Model-Based Development – Simulation of on-road vehicles scenarios is essential to validation of self-driving vehicles. Developing software to simulate real-life environments allows testing to be done on the computer rather than on the road.
Application Lifecycle Management – ALM encompasses the methods and processes through which software is developed, managed, and controlled. A well-defined ALM system ensures that the development team is efficiently working toward a common goal and that the end user receives software suited for the purpose intended.
Test Systems – With millions of lines of code in AVs, establishment of automated testing systems and processes will be crucial considering the safety-critical environment.
Analytics – Vehicles communicating with each other and back to the design team will produce large amounts of data. Analytics incorporates the storage and interpretation of data and identification of consequential patterns.
Summary
Mankind can gain value from AVs, but only if the public perceives that the benefits outweigh the costs and potential hazards. Trust will be central to public acceptance.
To gain that trust, the industry must understand the characteristics of trustworthiness and should align on an ecosystem that can produce vehicles worthy of trust.
ISO 26262 is an evolving standard for automotive development. Read how recent changes to the standard impact traceability, risk management, validation & verification in this joint white paper from Jama Software and LHP, “The Impact of ISO 26262 on Automotive Development.”
https://www-dev.jamasoftware.com/media/2019/10/Product_Development_Innovation-2.png5121024Steve Neemeh, LHP Engineering Solutions/media/jama-logo-primary.svgSteve Neemeh, LHP Engineering Solutions2019-10-22 08:35:302023-01-12 16:51:20Trustworthiness and the Autonomous Vehicle with the LHP Ecosystem
Few industries are as visibly affected by digital transformation as the automotive industry. Over a century ago, Henry Ford started to produce the production of the famous Model T and jump-started the success of the car as a consumer item. There has been lots of incremental innovation ever since, but nothing compared to what we are seeing today, like self-driving cars, over-the-air feature enablement, or transportation as a service.
Challenges for New and Established Organizations
It is unclear who is in a better position for exploiting the exciting opportunities in the automotive market. Established vendors have a solid, financially-strong foundation to operate from, combined with decades of experience in building cars. But this legacy is also slowing innovation down, as manufacturing processes and production lines are based on long development cycles. There are reasons for it: Existing automotive design processes evolved from a paper-based mindset, where every additional iteration in is expensive. Likewise, the production line was tuned for high throughput at low per-unit prices, but this makes changes very expensive.
New players in automotive engineering, on the other hand, are generally more agile and open. They can react faster and are willing to think outside the box. But their lack of experience can result in major issues with respect to compliance and production, creating expensive delays at late stages in development.
The First Generation of Requirements Management Tools
For decades, the automotive industry had recognized requirements as a key factor for success. The German automotive industry, in particular, started to introduce dedicated requirements tools in the ‘90s, and they encouraged their suppliers to do the same. Their solutions were establishing a new paradigm, using an item-based data model, rather than documents. This means that every requirement was an atomic item, with its own attributes, like priority and version history.
IBM® DOORS® (IBM Engineering Requirements Management – DOORS Family) was pioneering the idea of item-based requirements, and until the mid 2000s, there was really nothing like it available on the market. But by that time, some of the shortcomings of DOORS were becoming apparent. As a result, requirements specifications with tens of thousands unstructured requirements were not unusual, leading to redundancies and inconsistencies.
As a result, the first generation of requirements management tools were effective in helping the industry to scale. To a degree, they also helped in managing complexity. But they did not help in making the industry more innovative.
See how one Fortune 100 semiconductor company is managing the complexity of automotive development in our whitepaper. Read now.
Innovating with the Current Generation of Requirements Management Tools
Jama Connect™ was created to address the shortcomings of the first generation of requirements management tools. Specifically, Jama Connect was designed to solve the top challenges faced by regulated systems development teams. This fits the situation in the automotive industry today, and the following capabilities are key enablers for innovation:
Ease of Use – Requirements are used by everyone involved in automotive engineering. Not just the development team, but also engineering, quality, product marketing, sales, and leadership rely on requirements for an up-to-date product description. Good requirements also help departments make informed decisions. By contrast, first-generation tools were so hard to use that many important stakeholders refused to use them.
Single Source of Truth – Innovation requires trade-offs, which must be based on facts. It is crucial that facts — possibly derived from various sources — are instantly available. The first generation of tools typically only held a fraction of key data, and generating meaningful reports was a tedious, manual process.
Collaboration – Innovation requires heavy collaboration. First-generation tools did not have collaboration features, which meant that it took place outside the tool, i.e. via email or in meetings. But that creates a lot of friction and leads to situations where important information is missed. Modern solutions, by contrast, allow collaboration in context. This means that you have visibility into all relevant data and access to all relevant stakeholders while collaborating. And not just that…
Audit Trail – While first generation tools already had proper versioning, modern solutions go much further: For instance, the context-based collaboration leaves an audit trail in the tool (rather than in Outlook). This is crucial for efficient risk management and compliance. Developing new, innovative products typically results in much more activity. Context-based collaboration records decisions, issues, and answers exactly where you need them for compliance, and where you find them when you make reviews and trade-offs.
Learn how to avoid some of the biggest mistakes while developing automotive products for ISO 26262 compliance. Read the guide.
Managing Change – More than anything else, innovation requires the ability to react to change, no matter whether from the outside (new market needs) or inside (new technologies). The first generation of tools already acknowledged that by offering traceability-based functionality. But it was cumbersome to create and maintain the traceability, and many traceability-related issues were not apparent during day-to-day work but required running special reports. Modern solutions, by contrast, offer actionable traceability: Gaps in coverage or items that are marked as suspect due to change are visualized in many places, and allow fixing with a few clicks. This takes most of the friction out of the management of the traceability and allows for faster development iterations.
Reuse – Ironically, using what we already designed is an enabler for innovation. Without effective reuse, a lot of time and energy is wasted on re-designing what had already been designed and to re-test what had already been tested. It allows to focus our energy on the 20% that are truly new and truly innovative.
Integration – Last, the first generation of requirements tools created internal silos by making it very hard to get data in or out. Large vendors often offered half-hearted integration with their own software tools. While this was better than nothing, it often resulted in mediocre tool chains that were not based on need, but on availability. Jama Connect, by contrast, embraces openness by offering powerful interfaces that encourage integration with best-of-class tools.
Innovation in automotive engineering is only possible if all stakeholders can collaborate effectively, even in the context of very complex products, without being held back by the need to collect data from multiple places, or burdensome regulatory overhead. This is only possible with current generation requirements tools like Jama Connect.
*IBM® and DOORS® are registered trademarks of IBM Corporation.
Learn how the combination of Jama Connect and our Automotive Services tightens your development process by reading our datasheet.
https://www-dev.jamasoftware.com/media/2019/09/Blog-Legacy-Part-3.png5121024Michael Jastram/media/jama-logo-primary.svgMichael Jastram2019-09-17 04:30:592023-01-12 16:51:25Legacy Sunset: Enabling Innovation in Automotive Engineering Through Modern Requirements Management
For decades, companies building highly regulated, technically complex products and systems have relied on legacy tools like IBM® DOORS® or even Microsoft® Office® for requirements management (RM).
As software becomes more prevalent in physical products and development methodologies evolve, legacy RM tools like these have struggled to keep up with the modern speed of capturing, sharing, and managing requirements.
See how Jama Connect stacks up to legacy software tools like IBM® DOORS® in our whitepaper.
Legacy tools are powerful, but even with their complex capabilities and reputation for stability, they don’t always match the goals of teams who need to adapt, innovate, and grow.
Noticing this misalignment can surface as painful moments — like when your team struggles to get work done. Sometimes those issues are process or market related, but in some cases they are actually because of the exact software tools you’re using to make your life easier with requirements management.Here are some critical signals that your team is no longer aligned with your legacy solution for requirements management, and how you might consider a change going forward.
Painful Moment 1: You value the input of multiple roles and skillsets, but you realize only a handful of people know how to use your RM tool (or worse, they’re not allowed into it).
Why It Matters: This is a signal that the RM tool was not set up with the whole team in mind. For companies innovating quickly, this lack of visibility into what you’re working on and why is a major risk. If it were possible for only a handful of people to build complex products, teams and companies wouldn’t have the diversity of skills and thought processes they do today.
What to Do: Consider a system that supports the workflow of multiple roles with advanced, user-friendly tools for different data types and views that still all roll up to the same goals. User-friendly traceability is essential.
Painful Moment 2: You miss the deadline for feedback on a requirement because the notification was unclear. Now you have to go outside the normal channels to have your input seen.
Why It Matters: This is an instance not always appreciated by those who wrote the processes. However, with legacy software that doesn’t make the intentions of users clear (such as wanting input by a specific date), it’s hard to avoid miscommunication and lost time.
What to Do: Look for RM tools that make the next action required of you clear and sends timely notifications with clear instructions.
Painful Moment 3: You try to enter data into the system, but it’s locked, archived, or somehow just not visible.
Your decision is blocked. The person you know who provides access and training is unavailable (for example, on vacation… or no longer at the company).
Why It Matters: Having company-mission critical software that only a handful of people understand and have access to is a huge risk for many reasons – some not as obvious. On the lower end of severity, you lose a few hours trying to access the right data. However, if the trend is that fewer people have permissions and training to use your RM tool, then you’re at risk of data being lost or your system eventually needing to be replaced entirely.
What to Do: Data can get richer over time; it doesn’t have to go stale. Invest in requirements management tools that support your team’s continuous growth and capture continuous input and collaboration.
Painful Moment 4: Your team wants to upgrade its legacy software to fix issues, improve security, and access the latest features. Unfortunately, it’s so customized and patched together that you can’t update without major consequences.
Why It Matters: The products you’re building change and improve over time. The software tools you use to get your work done every day need to keep up with you, at a pace that balances change with allowing users time to learn.
What to Do: Consider the risk and costs associated with stale software, and look for options that avoid getting you locked in to unsupported versions and brittle customizations.
There’s a lot of requirements management solutions on the market. Cut through the clutter with our buyer’s guide.
Painful Moment 5: You want to change the way your team uses the legacy software to match a new process, but there are no configuration experts available.
What if you break something? You hope you can create an experiment project on the side that will ultimately integrate back into the big picture, but it’s not clear how to create this outcome.
Why It Matters: If teams are struggling to get on the same page with their requirements management process, it’s a signal that whatever your building may have misalignments as well.
What to Do: Consider RM solutions that balance each team’s need for autonomy with the need for alignment. This can be supported by software that have services experts available to adapt the tools to your needs as you change, rather than one big rollout that attempts to predict the future.
It’s tough to have to work around legacy software to get your job done. It’s also expensive. Maybe you can’t change the RM tools you use on demand, but if you start to notice these painful patterns it becomes easier to build a business case for making a change that supports the whole company’s vision.
https://www-dev.jamasoftware.com/media/2019/09/Blog-Legacy-Part-2.png5121024Robin Calhoun/media/jama-logo-primary.svgRobin Calhoun2019-09-11 04:30:172023-01-12 16:51:26Legacy Sunset: How to Stop These Five Legacy Software Pains in Requirements Management
Today, we’re re-running this post from January to help spread the word about our upcoming webinar that dives deep into the Engineering.com report. Register here.
A new research report from Engineering.com reveals how design and engineering professionals are approaching the increasing complexity of both their products and requirements management processes.
For the report, Design Teams: Requirements Management & Product Complexity, Engineering.com surveyed nearly 250 design and engineering professionals about the growing complexity of their company’s products and how requirements are managed amidst this changing landscape.
For a full breakdown of the Engineering.com report with guidance on how to correct some of the issues raised, register for our webinar, “The Great Cost of Poor Requirements Management.”
While Jama Software sponsored the report, the research and its subsequent editorial readout were conducted entirely independently by Engineering.com. And the results are extremely illuminating.
Product Development Increasing in Complexity
One of the biggest takeaways, especially for requirements enthusiasts like ourselves, was that even though more than 90% of respondents reported that their products had increased in complexity over the last five years, a mere 15% relied on a dedicated requirements management platform.
Driving this increase in product complexity are many factors, including how development teams have been:
Integrating more electronic components, embedded software and microprocessors
Increasing the intricacy of mechanical designs
Using different materials
Reducing weight and shrinking size
Building connected products (IoT)
Incorporating AI or machine learning
Given the rapid increase in product complexity, product design teams need a structured information system to handle requirements throughout each stage of product development.
Without a dedicated solution, teams are stuck with ineffective requirements management, resulting in things like product outcome failures (83% of respondents) and reprimands by regulatory agencies (62% of respondents).
The Impact of Investing in Requirements Management
Despite a high percentage of teams encountering regulatory and product outcome failures, most product teams (81% of those interviewed, in fact) believe their requirements management process to be effective.
And yet how can these requirements management processes be considered effective when the majority of teams are experiencing product outcome failures and getting penalized by regulatory agencies?
The reason for this contradiction in perception, according to Engineering.com, might be that “few teams employ a formal requirements management system. Having not purchased a system, they are dismissive and forgiving of failures while offering middling praise for the unsophisticated solutions they rely on.”
In other words, most companies don’t know what they don’t know.
Requirements Management Platforms Can Help Reduce Regulatory Reprimands
It shouldn’t be a surprise that development teams in industries that were highly regulated (66%) were more likely to use a purpose-built requirements management solution.
Perhaps not coincidentally, the analysis also showed that organizations using dedicated requirements management platforms in regulated industries not only received fewer instances of warnings, recalls, fines and production stoppages than those that didn’t, but nearly half reported experiencing none of those problems at all.
Plus, across all industries, requirements management platforms were rated highest in effectiveness overall versus any other method.
As the Engineering.com report concludes: “It is clear from the data that the products most design teams are creating are becoming more complex. Yet, they have not thought of investing in the tools available that would help them manage the requirements this complexity demands.”
Here are some other takeaways from the report:
Product teams must manage multiple types of requirements. On average, product teams cited 4.5 different requirement types critical to their products.
Product feature requirements are critical to 79% of respondents (the other 21% were working on less-complex products with fewer feature requirements).
More than 4 out of 5 teams have experienced product outcome failures, including exceeding cost requirements (46%), lost time to market (38%), product shipped without meeting all requirements (36%), and excessive time spent tracing requirements (36%).
The more sophisticated the requirements management system, the more effective it was found to be.
https://www-dev.jamasoftware.com/media/2019/01/Engineering.com-Jama-Software.jpeg5121024Jama Software/media/jama-logo-primary.svgJama Software2019-08-26 06:00:522023-01-12 16:51:28Webinar Preview: The High Cost of Poor Requirements Management
