MODELING OF URBAN POLICIES FOR HOUSING WITH FUZZY COGNITIVE MAP METHODOLOGY

PhD Lecturer at “School of Computer Science for Business Management” – Romanian-American University

)

JISOM Author: MARILENA – AURA DIN [1]*, CRISTINA COCULESCU [2]

ABSTRACT

Considering the increasingly dynamic and complex reality of the different urban policies, arose the question of whether these policies should be better designed in accordance with the causal knowledge in the respective policy area. One of the current approaches in urban policy modelling is the Fuzzy Cognitive Maps (FCM) methodology, which is capable to capture and to represent complex relationships in an intuitive manner and to represent the causality and the propagation of causality in the dynamic systems.

The purpose of this paper is to illustrate how the FCM methodology is useful for a cognitive modeling to various urban policies in the field of the housing and how the FCM can become a useful tool in the design of participatory urban policies based on the most important determinant factors (controllable, observable and influence variables) perceived by participants in the policy (citizens, city officers, experts in the field, etc.).

KEYWORDS: Housing policy modelling, simulating scenarios, Fuzzy Cognitive Maps (FCM).

JEL code: C6

  1. Introduction to urban policy modeling

By modeling, the policies can be designed as instruments capable to be experimented by scenario designed before putting them in practice, and also the future can be predicted after the policy implementation. Traditionally, the policy modeling refers to empirical analytical scientifically research by using of different quantitative or qualitative models and techniques to evaluate the cause and effect of any policy implication on the society [1].

Considering the increasingly dynamic and complex reality of the different urban policies, arose the question of whether these policies should be better designed in accordance with the causal knowledge in the respective policy area. In other words, it is necessary for decision makers to rely on intelligent tools with visual features combined with e-governance tools and techniques, and advanced ICT technologies such that to provide the necessary information to design generic models of knowledge in a certain field of politics.

The modeling of different urban policy categories has been approached among others scientifically methods and techniques by regression analysis, Markov Chains, Colored Petri Nets (CPNs), Multi-Agent Simulation (MAS), Cellular Automata, etc. In order to take into account the causes of the behavior of the actors in the different urban policy domains, we introduced the Fuzzy Cognitive Map (FCM) methodology which is said to be capable to deal with the causal and cognitive modeling of the participants’ behavior in the urban policy. Regarding urban policy modeling, Jean-Luc de Kok [5] is among the few people who applied FCM to predict urbanization.

  1. Introduction to Fuzzy Cognitive Maps

Fuzzy cognitive maps (FCMs) are a soft computing methodology introduced by Kosko in 1986 [4] as an extension of cognitive maps, that use the cognitive knowledge of human experts on specific domains by creating causal models as signed/weighted directed graphs of concepts and the various causal relationships that exist between those concepts (Axelrod (1976), Kosko (1986)). The previous Cognitive Maps (CMs) have been used successfully in the past in the different situations of decision making[3] (Ross, 1976), prediction[4] (Robert, 1973), explanation[5] (Bonham and Shapiro, 1973) or strategic planning [6](Hart, 1976; Tsadiras, Margaritis and Mertzios, 1995).

FCM is a causal graph that includes the following components:

(a) Nodes: Each concept node indicates a characteristic of the system such as events, actions, and states.

(b) Edges: to represent the direct causal influence between concepts.

(c) Weights: to represent how much one node influence another.

(d) Activation events at different moment t. The stimulated events can bring changes to certain concepts, edges, or even the overall of FCM.

FCMs integrate the cognitive maps of accumulated experience and knowledge concerning the factors and the underlying causal relationships between factors of the modeled system. The decision-makers’ maps can be examined, compared as to their similarities and differences, and discussed [3]. FCMs are designed by experts through an interactive procedure of knowledge acquisition.

In this paper we illustrated how the FCM methodology is useful for cognitive modeling to various urban policies in the field of the housing and how the FCM can become a useful tool in the design of participatory urban policies based on the most important determinants (controllable, observable and influence variables) perceived by participants in the policy (citizens, city officers, experts in the field, etc.).

  1. Housing policies – Questions under research and rationales

One of urban policies of great interest is the housing policy. Within housing policies, there are several categories of interest, such as Housing Ownership, Housing Choice, Neighborhoods Services and Amenities, Housing Availability, Neighborhood Stability, Leveraging Resources, and Housing Quality. For these categories of housing policy, we list below some questions under research and the rationales for the selection of the most important determinant factors in the field.

Table 1. Questions under research and the rationales for the selection of the most important determinant factors

Housing Policy and The question under research Rationales for the selection of the determinant factors and causal relations in the field
1. Housing ownership policy: Is it appropriate or of interest to own the house you are living in? The option for ownership can cause changes/shifts in ownership of property: owner or tenant and hence new housing policy design. Changes caused by the preferences for owner or tenant option could enhance of financial support to increase rental housing stock or regulatory measures to purchase the property rented by tenants who occupy them, such as interest subsidy, loan guarantees, etc.
2. Housing choice: Which is the preference of citizens to relocate for living in the city If the interest expressed by opinion questionnaires for relocation in the city is increased, then will increase demand for houses and, depending on the volume of supply of existing housing stock, will increase or not the level of taxes.
3. Neighborhoods services and amenities: How much is the opportunity to create and support a variety of services and facilities to offer Increase in diversity and number of services and facilities offered in neighborhood affects diversity of population. Increasing diversity of services such as recreation, trade, education, information services, laundry, tailoring, business centers, communication services, transportation services and so on, attracts population with different leisure preferences, different education levels, different occupations and interests, which in turn will influence the job market in neighborhood area. Increasing in services offered in the vicinity of the house also lead to improve standards of living conditions, therefore the health and welfare of the population, which means social effects of the policy.
4. Housing Availability: How do you describe the housing needs of special population groups? Housing availability looks closely at the housing needs of special population groups. The special groups addressed in this section include ex-offenders, the homeless, persons with mental or physical disabilities, as well as the needs of youth and seniors. While these needs often overlap with the affordability principles, there are additional issues to consider including location, special physical accommodations, and availability of supportive services. Special physical accommodations and supportive services such as training and counseling public services could also increase the housing availability.
5. Neighborhood Stability: How do you appreciate/evaluate the conditions of neighborhoods? 

 

The neighborhood stability policy considers an underlying principle related to housing – and that is the neighborhood in which housing exists. A housing policy would be incomplete without considering the condition of neighborhoods – whether or not there are blighted properties, paved streets and sidewalks, and proximity to schools and shopping, etc. Crime and code violations are additional neighborhood considerations which are addressed.

With more investments in conditions assurance and in infrastructure, the improvements are gained in neighborhood stability.

6. Leveraging Resources: How leveraging resources could generate income for housing policy?

 

As its title suggests, leveraging resources issue is based upon the premise that there is no single financing structure that will achieve all of the goals of the housing policy. In this respect, single and multi-family housing initiatives deserve to be explored and recommended to housing policy. Housing is a basic human need and right – and beyond the human implications, it is a tremendous community economic engine.
7. Quality: Which are the principal factors that characterize the housing quality?

 

The issue of Quality – of housing and neighborhoods relates to housing which is desirable, functional, safe and energy efficient. Community commitment for quality environment is in meaningful high quality housing in high quality neighborhoods – regardless of the physical location. These principles of quality are important considerations for future homebuyers or renters, when they make choices as to where to live.
8. Housing Affordability: How do you consider is the access to affordable housing? Housing affordability (to buy, rent or build) is affected in principal by household incomes, housing costs, supply of housing, and the cost of borrowing money. It is also much localized (i.e. what is affordable in a city is unheard of in another one). Housing affordability policy means the ability to select the location and type of housing that a household can ‘afford’.

     In the next parts of the paper we refer to one of these housing policies, exactly to the housing affordability, for which we will apply the FCM techniques, firstly to identify behavioral dependencies.

  1. The case of modelling a housing policy with FCM Methodology

The mathematical programming-based models for affordable housing policy design are using observations provided by the community-based nonprofit housing developers [2]. Here we give an example of applying the FCM methodology for the cognitive model of the housing affordability policy.

Our research includes examining the perceptions of different stakeholders groups (representing households, affordable housing providers – public housing authority, municipal housing administrators, and private housing builders) about housing affordability. Cognitive mapping methods are especially designed for systemic approaches and can thus make visible previously unknown and surprising effects of the system [6].

Based on experience and knowledge of the system under consideration, responsible person with design of the cognitive map, decides the important factors that affect a system and then draws causal relationships among them indicating the directions of the causal relationships with arrowheads, and the relative strength of the relationships with a number between −1 and 1. More simulations are done to learn how the model changes with changing strengths of relationships.

A FCM works in more discrete steps [3]. Present methodological approach of applying FCM for Housing Affordability domain consists of distinctive parts as follows:

  1. Selection of factors and causal relations,
  2. Stakeholders interviews,
  3. Coding the cognitive maps into adjacency matrices,
  4. Drawing the FCM,
  5. Selection of factors and causal relations

The relevant factors that will represent concepts in the cognition map, and the relationship between them for the field of housing affordability are grouped in three categories, as in the Table 2.

Table 2. Examples of factors and causal relations

Categories of determinant Factors Causal relationships – description
Population  
Households income Bigger income of a buyer gives him more affordability to acquire a house.
Financial literacy of buyers Poor personal financial literacy leaves people, especially low-income people, vulnerable or being turned down for a mortgage.
Offers of lenders Lenders who may or not offer low cost financing for families in need.
Community attitudes One obstacle to affordable housing in suburban locations is the resistance of many homeowners to permitting low-cost housing within their neighbourhoods might threaten appraisal value of their homes
Economic  
Housing costs Housing costs are defined as rent or principal, interest, taxes and insurance combined.
Costs of financing Costs of financing influence the purchase transaction (include mortgage rate, length of the loan, points, and closing costs). Change any of these factors and the down payment or monthly payments could go up or down. A home may be affordable to own, but with much up-front cash required, which means at last also unaffordable to purchase
Costs to build houses Builders of new homes typically operate on fairly narrow net profit margins, so even a small spike in costs of materials can cut drastically into a builder’s profit and increase housing costs to buyers. Even innovations in materials and methods, the independent builder is generally not able to have much of an impact on overall housing affordability.
House building Builder groups often claim—and government statistics support these claims—that home building traditionally leads the nation out of recession.
Trades;Manufacturing Professional services; Transportation Home building benefits not only the trades but also manufacturing, professional services, and even transportation.
Job market Developing of job market attracts increase of housing demand.
Demand for new housing The increase and diversification of job market in neighborhood could influence the household income and therefore the housing affordability.
Social-political
Land use regulations One of the key ways to bring housing costs down is to increase housing density if land use regulations allow. Land use regulations allow housing affordability. There are certain fixed costs to developing any parcel of land including site planning and permits, roads, power, sewer, and water. All of these costs have to be included in the selling price of the housing that is built on the parcel.
Public services for training and counselling Special physical accommodations and supportive services such as training and counselling public services could also increase the housing affordability for families in need of credit counselling.
Educational programs Educational programs available to assist individuals for families in need of credit counselling can lead to homeownership and also teach them how to take care of themselves financially.
Local legislation A decrease level of taxes, or subsidies for support families in need. There are also a number of non-profit organizations and programs that specifically address housing affordability, most of which take advantage of available government subsidies for construction and homebuyer assistance. (subsidizes social programs for first time home buyers, land facilities for construction, distance from house to job for essential workers)
Natural changes
Weather Delays due to weather or natural disasters issues also add to costs and these costs get passed on to the buyer
Natural disasters

 

  1. Stakeholders interviews

Generally, the cognitive maps can be obtained in different ways: from questionnaires, by extraction from written texts, by drawing them from data that shows causal relationships or through interviews with people who draw them directly.

Here the purpose is to illustrate how to obtain the views of the different stakeholder groups, their differences and similarities about housing affordability using interviews like in the interview guideline in the table 3.

Table 3. Interview guideline for creating the FCM for housing domain

Interview steps/target

Research questions

Interview questions How to conduct
To establish the housing policy theme

„Experiencing living in this city, what is important for you regarding housing?”

„Do you prefer to relocate for live in or out the city?”

„Do you afford to buy/rent a house?”

„Is it appropriate or of interest to own the house you are living in?”

„Are you satisfied with the neighborhood’s services and amenities?”

„How important are the housing needs of special population groups?”

„How do you appreciate / evaluate the conditions of neighborhoods?”

„How important are the leveraging resources could generate income for housing policy?”

„How important is the housing quality?”

Ask the interviewees to order the importance of the  housing categories:

–       Residential location choice

–       Housing Affordability

–       Housing ownership

–       Neighborhoods Services and Amenities

–       Housing Availability

–       Neighborhood Stability

–       Leveraging Resources

–       Housing Quality

 

To find what is perceived as important factors/concepts in the domain of housing, when considering the theme chosen above „What do you see as capable of changing the housing picture?”

„What have changed with housing since you started coming here?

The interviewees are asked to list on the paper all concepts identified through previous answers

 

To draw the connections between the identified concepts „Write the most important concept in the middle of a large drawing paper”

“Draw the lines that link all of these concepts which affect each other.”

“Are there any concepts that affect the concept [X]?”

“Does this concept [X] affect any other concepts?” such as

The interviewee is asked to pick one of the concepts of the list e.g. the one which is considered most important and to draw lines between them in order to represent the relationships / connections that link each of them.

 

To indicate the sense of directions of the connections from causal concepts to affected concepts „When considering X, concept, what other concepts affects X?” The interviewee is asked to point the arrows on drawing lines to indicate the sense of direction from the causal concepts directed to the affected concepts, as they were identified by previous questions
To establish the positive or negative influence between  concepts

 

“How affects these concepts each other (positively, negatively, feed-back mechanisms)?”

„For this purpose, label the lines with signs of positive, or negative.”

The interviewee is asked to assign “+” to the positive connections and “-“ to the negative ones
To determine the strengths to the connections – to assigne of the linguistic weights to causality relationships „How strongly the concepts affect each other?”

„How strength is this positive /negative effect on X (large, medium, small)?”

„How much does it mean for X that Y changes (much, little, in between)?”

The interviewee is asked to weigh the influences between concepts, and to assign to each arc a linguistic weights, such as “strong”, “weak”, or “lack” “low”, “medium”, “high” etc.

Thus, the degree of causality among the concepts is qualitatively described by linguistic weight, subjective given by words.

 

Bodies of knowledge considered important for a housing policy could be classified as traditional knowledge, local knowledge or expert knowledge. This could be the reason for interviews to be organized on homogeneous groups representing the beliefs and perceptions of stakeholder groups like households, enterprises, representatives of public authorities, housing developers, estate agents, and land owners. In sessions with homogeneous groups, the participants are assisted by a facilitator who can debate the difficult issues and create a common understanding of the problem with consequences in exclusion or inclusion of different factors.  Aggregation can also be used to reduce the complexity of large maps.

Different individuals inside each group create FCMs with a sample size. Adequate sample size for maps / interviews can be determined by examining the accumulation curves of the total number of concepts/variables versus number of maps/interviews as well as the number of new concepts added per map/interview. Average accumulation curves can be made by using Monte Carlo techniques. Since human experts are subjective and can handle only relatively simple networks (maps), there are necessary to apply different methods for automated generation of FCM models, such as the learning method based on genetic algorithms introduced by Stach et al., (2005)[8].

  1. Coding the cognitive maps into adjacency matrices

After the interviews, the Cognitive Maps are transformed into matrices in the form (Wij)ij  [7]. The linguistic variables that describe each arc, for each interviewed are characterized by the fuzzy sets. The linguistic variables are combined, and the aggregated linguistic variable is transformed to a single linguistic weight, through the SUM technique (Lin and Lee 1996). Finally, the Center of Area (CoA) defuzzification method [4] is used for the transformation of the linguistic weight to a numerical value within the range [−1, 1].

The concepts Ci (e.g housing costs) are listed on the vertical axis and Cj (e.g., housing affordability for buyers) on the horizontal axis to form a square matrix (Wij)ij  where Wij takes any value in the range −1 to 1, based on the cognitive maps. The element in the ith row and jth column of initial weight matrix (Wij)ij  represents the strength of the causal link directed out of node Ci and into Cj. For example, Wij =-1 is entered if there is a causal decrease from Ci to Cj (e.g., housing costs decreases housing affordability for buyers).

The new value of any concept is calculated based on the current values of all the concepts, which exert influences on it through causal links. This computation of a node’s output is based on the combination of a summing operation followed by the use of a non-linear transformation function such as threshold function.

The value of a concept, Ci is derived by the transformation of the fuzzy values to numerical values. Since the values of the concepts, by definition, must lie within [0, 1], the chosen function f is regularly the sigmoid function. Typically, the activation function is the logistic function: F(x)= 1/(1 + e−x) , that transform the results into the interval (0, 1).

To receive information on the dynamic behavior of a FCM we have to calculate the influence one factor has on others over a number of iterations (the feedbacks between the concepts). The computation of the Cj node’s output is given by formula:

MODELING URBAN POLICIES HOUSING FUZZY COGNITIVE MAP METHODOLOGY 1 MODELING OF URBAN POLICIES FOR HOUSING WITH FUZZY COGNITIVE MAP METHODOLOGY

where k is the iteration counter; and Wij is the weight of the arc connecting concept Ci to concept Cj. After a number of iterations FCMs will either converge to a stabile state, implode (all factor values converge to zero), or explode (all factor values increase /decrease continuously) or show a cyclic stabilization.

Having assigned values to the concepts and weights, after a few iterations the FCM converges to a steady state. At each step, the value of a concept is influenced by the values of concepts–nodes connected to it, till the system would converge to a point and no further changes would take place. So, a FCM can simulate its evolution over time to predict its future behavior.

  1. Drawing Fuzzy Cognitive Maps

To visualize the FCM for the example of housing affordability, we have used the software FCMapper (http://www.fcmappers.net).

The following figure is the graphical representation FCM for Housing Affordability, for 106 total numbers of connections corresponding to 22 nodes:

MODELING URBAN POLICIES HOUSING FUZZY COGNITIVE MAP METHODOLOGY 2 MODELING OF URBAN POLICIES FOR HOUSING WITH FUZZY COGNITIVE MAP METHODOLOGY

Fig.4 FCM for Housing Affordability

  1. Simulating scenarios

The use of FCM modeling to simulate different housing polices scenarios offers a convenient way to experiment with policy alternatives.

As one of the main advantages of FCM is that it offers a way to involve stakeholders in participatory modelling or scenario projects, or it can be used as a stand-alone tool to develop scenarios such as:

– Scenario for various types of affordable/subsidized housing units,

– Scenario to simulate the dynamic model of affordable housing demand based on features of the housing market including costs from buying and selling a home, credit constraints and uncertainty about the evolution of incomes and home prices, in order to simulate how consumer behaviour responds to house price and income declines as well as tightening credit,

– Scenario assuming chosen configurations of affordable housing across a study area to minimize total costs while accounting for social impacts (maximize net social welfare, subject to a budget constraint and to maximize a measure of equity or fairness, e.g. minimize the variance in the number of housing units allocated to target neighborhoods across the study area), etc.

Through policy option simulations, it is possible to determine which combination of policies would increase the housing affordability, according to people perceptions.

Given an initial state of the system, represented by a set of arbitrary values of its concepts, a FCM can evolve over time until a state of equilibrium, i.e. until it reaches the steady state. This steady state can be used to make different scenarios. Fine modifications of one or several factors in the equilibrium state will yield to different comportments of the system.

Two scenarios were imagined in our model: first, we diminish the Land use factor from 1 to 0.5, second we have increased the same factor to 0.7.

Comparing the scenarios with the steady state, we’ve obtained the following conclusions listed in table 4 for in each scenario, where the change of the link strength is shown:

                        Table 4. Results of the simulated second scenario

Positive Changes strength (+) Negative Changes strength (-)
Offers of lenders high change Housing affordability no change
Community attitudes high change Households income medium change
Housing costs no change Level of education low change
Costs of financing low change Financial literacy low change
Neighborhood facilities medium change House building high change
Subsidy medium change Trades high change
Manufacturing high change
Professional services medium change
Transportation high change
Jobs market high change
Housing demand high change
Builder’s profit medium change
Local legislation no change
Training and counseling medium change
Subsidy no change

Comparing the second simulated scenario with the steady state, we have reached the following conclusion listed in table 4: The change of the Land use factor will lead to strong positive change in Offers of lenders, and Community attitudes, medium positive influence on the Neighborhood facilities, strong negative change on House building, Trades, Manufacturing, Transportation, Jobs market, and Housing demand, but no change on Housing Availability.

As a tool to further develop various types of scenarios, FCM can be used in order to simulate how consumer behavior responds to house price and income declines while accounting for social impacts.

 

  1. Conclusion

Considering the FCM’s potential to be used in the policy modeling, and representing the causality and the propagation of causality in the dynamic systems, this paper explores how FCM can be applied to the particular urban policy of housing affordability.

The FCM results for housing affordability concerns structural and dynamical analysis:

– Structural analysis use comparative statistical techniques based on comparing the structural indices values (density index, in degree, out degree, transmitter, centrality, hierarchy index, and complexity index) among the groups of stakeholders interviewed.

– Dynamical analysis use computational simulations. A vector of initial state of variables is multiplied with the adjacency matrix of cognitive map and the results are transformed to the (0, 1) interval using a logistic function. These steps are repeated through iterations while the matrix values become steady state. Then it is possible to run „what-if” scenarios, setting desired variables to a desired value and then comparing the outputs of the scenarios with the outputs of the base-line scenario.

Our conclusion is that in order to create a participatory urban policy for the exploration of the urban system complexity of housing, the FCM methodology is capable to evaluate the behavior of the system and his equilibrium states, and to determine critical factors affecting housing policy.

The advantage of such model is that it provides a better and more comprehensive understanding of citizen needs regarding housing while it offers a way to involve stakeholders in participatory modeling.

 

References

  • Estrada, R. Arturo M., The Difference between Economic Modelling and Policy Modelling (September 26, 2011) Faculty of Economics – University of Malaya Available at SSRN: http://ssrn.com/abstract=1933668.
  • Johnson, M.P., Economic and statistical models for affordable housing policy design. Working paper, Carnegie Mellon University, Heinz School of Public Policy and Management, Pittsburgh, PA, 2007.
  • Özesmi, U. and S.L. Özesmi. 2004, Ecological models based on people’s knowledge: a multi-step fuzzy cognitive mapping approach, Ecological Modelling, 176(1-2), pp. 43-64.
  • Kosko, B. 1986, Fuzzy cognitive maps, International Journal of Man-Machine Studies, 1, pp. 65-75.
  • De Kok, J.L., M. Titus, and H.G. Wind, 2000. Application of Fuzzy set and cognitive maps to incorporate social science scenarios in integrated assessments models: A case study of urbanization in Ujung Pandang, Indonesia, Integrated Assessment 1, pp. 177-188.
  • Isak K.G.Q., Wildenberg M., Adamescu C.M., Skov F., De Blust G., 2009. Manual for applying Fuzzy Cognitive Mapping – experiences from ALTER-Net, ALTER-Net Report.
  • Khan, M. S. and M. Quaddus. 2004, Group Decision Support Using Fuzzy Cognitive Maps for Causal Reasoning, Group Decision and Negotiation 13, pp. 463–480.
  • Stach, W. et al Genetic learning off Fuzzy cognitive maps. Fuzzy Sets and Systems 153 (2005) 371–401.
  • FUPOL Project financed by FP7 EU Program.
  • Din M.A., Moise M., „A Fuzzy Cognitive Mapping Approach for Housing Affordability  Policy Modeling”  – Latest Advances in Information Science & Systems Series/4, pp.262-267,

 

[1] * Corresponding author. Associate Prof, Ph.D., Romanian-American University, 1B Expozitiei Bd, Sector 1, 012101 Bucharest, ROMANIA, [email protected]

[2] Associate Prof, Ph.D., Romanian-American University, 1B Expozitiei Bd, Sector 1, 012101 Bucharest, ROMANIA, [email protected]

[3] Ross, S. A., 1976, Complexity and the presidency: Gouverneur Morris in the Constitutional Convention, In: Axelrod, R., Structure of Decision, Princeton University Press, Princeton, NJ, pp. 96-112.

[4] Roberts, F. S., 1973, Building and analyzing an energy demand signed digraph, Environment and Planning, Vol. 5, Issue 2, pp. 199-221.

[5] Shapiro, M. J. and Bonham G. M., 1973, Cognitive Processes and Foreign Policy Decision Making. International Studies Quarterly. No. 17, pp. 147–174.

[6] Hart, J., 1976, Comparative cognition: Politics of International Control of Oceans, In: Axelrod, R., Structure of Decision, Princeton University Press, Princeton, NJ, pp. 180-217. Tsadiras, A. K., Margaritis, K. G., Mertzios, B., 1995, Strategic planning using extended fuzzy cognitive maps, Studies in Informatics and Control, Vol. 4, Issue 3, pp.237-245.

Article reviewed and approved by: Garais Gabriel

PhD Lecturer at “School of Computer Science for Business Management” – Romanian-American University


Article Autor/s:

MARILENA - AURA DIN

Associate Prof, Ph.D., Romanian-American University, 1B Expozitiei Bd, Sector 1, 012101 Bucharest, ROMANIA, [email protected]

CRISTINA COCULESCU

Associate Prof, Ph.D., Romanian-American University, 1B Expozitiei Bd, Sector 1, 012101 Bucharest, ROMANIA, [email protected]

Leave a Reply